Annals of Mathematics, 149 (1999), 627-689 



Group cohomology and the singularities 
of the Selberg zeta function 
associated to a Kleinian group 

By Ulrich Bunke and Martin Olbrich 



Contents 

1. Introduction 

1.1. The Selberg zeta function 

1.2. Singularities and spectrum 

1.3. Singularities and group cohomology 

1.4. The main result 

1.5. The extension map 

2. Restriction, extension, and the scattering matrix 

2.1. Basic notions 

2.2. Holomorphic functions to topological vector spaces 

2.3. The push down 

2.4. Elementary properties of ext\ 

2.5. The scattering matrix 

2.6. Extension of hyperfunctions and the embedding trick 

3. Green's formula and applications 

3.1. Asymptotics of Poisson transforms 

3.2. An orthogonality result 

3.3. Miscellaneous results 

4. Cohomology 

4.1. Hyperfunctions with parameters 

4.2. Acyclic resolutions 

4.3. Computation of H*(T, O x C-"(A)) 

4.4. The r-modules {Aifc) C^(A) 

5. The singularities of the Selberg zeta function 

5.1. The embedding trick 

5.2. Singularities and cohomology 



628 



ULRICH BUNKE AND MARTIN OLBRICH 



1. Introduction 



Let G := SO(l,n)o denote the group of orientation-preserving isometries 
of the n-dimensional hyperbolic space X := H n equipped with the Riemannian 
metric of constant sectional curvature —1. We consider a convex cocompact, 
torsion- free discrete subgroup T C G. The quotient Y := Y\X is a complete 
hyperbolic manifold, and we assume that vol(F) = oo. 

The Selberg zeta function Zg(s), s G C, associated to this geometric 
situation encodes the length spectrum of closed geodesies of Y together with 
the eigenvalues of their Poincare maps. Note that Zs{s) is given by an Euler 
product on some half-plane Re(s) > c, and it has a meromorphic continuation 
to the complex plane. 

The goal of the present paper is a description of the singularities of the 
Selberg zeta function in terms of the group cohomology of Y with coefficients 
in certain infinite dimensional representations. Such a relation was conjectured 
by Patterson [37]. 

1.1. The Selberg zeta function. 

In order to fix our conventions we define Zs(s) in terms of group theory. 
Let g = k © p be a Cartan decomposition of the Lie algebra of G, where k is 
the Lie algebra of a maximal compact subgroup K C G, K = SO(n). We fix 
a one-dimensional subspace a C p and let M C K, M = SO(n — 1), denote 
the centralizer of a. The Riemannian metric of X induces a metric on a. We 
fix an isometry a = R. Let a + denote the half-space corresponding to the ray 
[0, oo) and set A := exp(a), A + := exp(a + ). By n C g we denote the positive 
root space of a in g. For H G a we set p(H) := ^tr ad(if)| n . The isometry 
a* = R identifies p with 

By the Cartan decomposition G = KA+K, any element g G G can be 
written as g = ha g k, h,k G K, where a g G A + is uniquely determined. We 
have a g = e dist (°^°), where O = K G X is the origin of X = G/K, and dist 
denotes the hyperbolic distance. The basic quantity associated with Y is its 
exponent 5r which measures the growth of Y at infinity. 

Definition 1.1. The exponent 5r G a* = R of Y is defined to be the 
smallest number such that the Poincare series 



(1) 




converges for all s > 5r- 



Although we do not use it in the present paper note that 5r + is equal 
to the Hausdorff dimension of the limit set of Y (see [35], [47]). 
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Any element g £ T is conjugated in G to an element of the form m{g)a{g) £ 
MA + , where a(g) is unique. By l g := log(a(<7)) we denote the length of the 
closed geodesic of Y corresponding to the conjugacy class of g £ T. 

Let CT denote the set of conjugacy classes [g] / 1 of T. If [g] £ CT, 
then nr(g) G N is the multiplicity of [g], i.e., the largest number k £ N such 
that [g] = [h k ] for some [h] £ CT. If A : V — > V is a linear homomorphism 
of a complex vector space V, then by 5 fc ^4 : S k V — > S*V we denote its fc th 
symmetric power. 

Definition 1.2. The Selberg zeta function Zs(s), s £ C, Re(s) > <5r, 
associated to T is defined by the infinite product 

oo 

(2) Z s (s) n II det (! - e-^'^^Ad^^a^))^ 1 )) . 

[g]eCr,n r (g)=l k=0 

Remark. We defined the Selberg zeta function such that its critical line 
is {Re(s) = 0}. In the literature the convention is often such that the critical 
line is at p = r± ^- . The same applies to our definition of <5r which differs by p 
from the usual convention. 

In [38] (see also [42]) it was shown that the infinite product converges for 
Re(s) > 5r, and that the Selberg zeta function has a meromorphic continuation 
to all of C. In the special case of surfaces this was also proved in [17]. Partial 
results concerning the logarithmic derivative of the Selberg zeta function have 
been obtained in [34] for 5r < and in [41] in the general case. 

1.2. Singularities and spectrum. 

The Selberg zeta function is a meromorphic function defined in terms of 
a classical Hamiltonian system, namely the geodesic flow on the unit sphere 
bundle SY of Y. Philosophically, the singularities of the Selberg zeta function 
should be considered as quantum numbers of an associated quantum mechani- 
cal system. One way to quantize the geodesic flow is to take as the Hamiltonian 
the Laplace-Beltrami operator Ay acting on functions on Y. 

In order to explain this philosophy let Y = T\G/K for a moment be a 
compact locally symmetric space of rank one. Then the sphere bundle of Y 
can be written as SY = T\G/M. If (a,V a ) is a finite-dimensional unitary 
representation of M, then we consider the bundle V{a) := T\G Xm V<j over 
SY. The geodesic flow admits a lift to V(a) and gives rise to a more general 
Selberg zeta function Zs(s,a) which also encodes the holonomy in V(a) of 
the flow along the closed geodesies. The Selberg zeta function Zs(s) defined 
above corresponds to the trivial representation of M. It was shown in [13] that 
Zs(s,a) admits a meromorphic continuation to all of C. In this generality a 
description of the singularities of Zs(s,o-) was first obtained by [21] (see also 
[48] for a closely related Selberg zeta function). 
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Remark. The case of a Riemann surface is classical. The Selberg zeta func- 
tions for general rank-one symmetric spaces and trivial a have been discussed 
in [14]. For a detailed account of the literature see [5]. 

The description of the singularities of Z$(s, a) given in [21] corresponds to 
a different method of quantization of the geodesic flow (see subsection 1.3). The 
spectral description of the singularities of Zs(s,a) uses differential operators 
acting on sections of bundles on Y. 

One distinguishes between two types of singularities, so-called topolog- 
ical and spectral singularities. If a is trivial then the spectral singularities 
are connected with the eigenvalues of the Laplace-Beltrami operator Ay. The 
topological singularities depend on the spectrum of the Laplace-Beltrami op- 
erator on the compact dual symmetric space to X. For general a we found 
the corresponding quantum mechanical system in [5]; i.e., we determined the 
locally homogeneous vector bundle on Y together with a corresponding locally 
invariant differential operator whose eigenvalues are responsible for the spec- 
tral singularities of Zs(s, a). The analytic continuation of this operator to the 
compact dual symmetric space gives rise to the topological singularities. 

We now return to our present case that T C SO(l,n)o is convex cocom- 
pact, Y is a noncompact hyperbolic manifold, and a is the trivial representation 
of M. It was shown in [28] that the spectrum of Ay consists of finitely many 
isolated eigenvalues in the interval [( !i ^-) 2 — |£r|#r> i 1 ^) 2 )- Moreover, in [29] 
the same authors show that the remaining spectrum of Ay is the absolute con- 
tinuous spectrum of infinite multiplicity in the interval [(^^y^) 2 , oo). It turns 
out that the eigenvalues of Ay are responsible for singularities of Z${s) as in 
the cocompact case. This stage of understanding is not satisfactory. On the 
one hand Zs(s) may have more singularities. On the other hand the continuous 
spectrum was neglected. 

A finer investigation of the continuous spectrum can be based on study of 
the resolvent kernel, i.e. the distributional kernel of the inverse (Ay — (^^) 2 + 
A 2 ) -1 . It is initially defined for Re(A) » 0. A continuation of this kernel up to 
the imaginary axis implies absolute continuity of the essential spectrum by the 
limiting absorption principle (see [39] and for surfaces also [12], [10]). But this 
kernel behaves much better. It was shown in [32] that it has a meromorphic 
continuation to the whole complex plane (for surfaces see also [9], [1]). The 
poles of this continuation with positive real part correspond to the eigenvalues 
of Ay. The poles with nonpositive real part are called resonances. 

Let us consider the resonances as sorts of eigenvalues associated to the 
continuous spectrum. Then they lead to singularities of Zg(s) in the same way 
as true eigenvalues. In detail, the spectral description of the singularities of 
Zs(s) was worked out in [38] for even dimensions n. 
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1.3. Singularities and group cohomology. 

An important feature of the spectral description of the singularities of 
the Selberg zeta function is the distinction between spectral and topological 
singularities. By now there are two approaches to describe all singularities of 
the Selberg zeta function in a uniform way, avoiding a separation of topological 
and spectral singularities. 

We will explain these approaches again for Zg(s,a) in the case where Y 
is a compact locally symmetric space of rank one. 

The first approach was worked out in [21] (ideas can be traced back to 
[16], [36]) and corresponds to a quantization which is different from the one 
considered for the spectral description. Here one considers the cohomology of 
the cr-twisted tangential de Rham complex (called tangential cohomology), i.e. 
the restriction of the de Rham complex of SY to the stable foliation twisted 
with V(a). This complex is equivariant with respect to the flow. The order of 
the singularity of Zs(s, a) at s = A is related to the Euler characteristic of the 
A + p-eigenspace of the flow generator on the tangential cohomology. 

The tangential cohomology comes with a natural topology, and it is still 
an open problem to show that this topology is Hausdorff. Therefore, in [21] the 
result is phrased in terms of representation theory, in particular in terms of Lie 
algebra cohomology of n with coefficients in the Harish-Chandra modules of 
the unitary representations occurring in the decomposition of the right regular 
representation of G on L 2 (T\G). 

The second approach was proposed by S. Patterson [37]. The parame- 
ters a and A € C fix a principal series representation (TT a ' X , H a > x ) of G. The 
space H a ' x can be realized as the space of sections of a homogeneous vector 
bundle V(a\) := G Xp V ax , where P := MAN, N := exp(n), and o\ is the 
representation of P on V a given by P = MAN 3 man ^ a p ~ x o~{m). 

Taking distribution sections we obtain the distribution globalization of 
this principal series representation which we denote by H^^. If V is a com- 
plex representation of T, then H*(T, V) denotes group cohomology of T with 
coefficients in V. The specialization of Patterson's conjecture to cocompact V 
is: 

(i) dmi/*(r,f£>4)<oo, 

(ii) x(I\l££) := E*=o(-l)* dim iT(r,iT^) = 0, and 

(iii) - X i(I\f£>4) := -£~ (-l)^dim W(T,H^) is the order of Z s (s,a) 
at s = A (a pole has negative and a zero has positive order). 

This conjecture was proved in [4] and [7] (with a slight modification for 
s = 0) by clarifying the relation between both approaches. In (i)-(iii) one can 
replace by the space of hyperfunction sections. 
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One way to define group cohomology is to write down an explicit complex 
(C*,d) such that H*(T,V) is the cohomology of this complex. One can take 
e.g. 

C p := {/ : rx^__T - V\f(gg , ...,gg p )= gf(g , ■ ■ .,g P )} 
p+i times 

and 

p+i 

(df)(g ,...,g p+ i) := ^(-l)7(c/ , • • • ,9i, ■ ■ ■ ,9 P +i) ■ 
i=0 

Alternatively one can define group cohomology as the right derived functor 
of the left exact functor from the category of complex representations of T 
to complex vector spaces which takes in each representation the subspace of 
T-invariant vectors. By homological algebra one can compute group coho- 
mology using acyclic resolutions. To find workable acyclic resolutions for the 
representations of interest is one of the main goals of the present paper. 

Let now r C G be convex cocompact such that Y is a noncompact mani- 
fold. The space G/P can be identified with the geodesic boundary dX of X. 
There is a T-invariant partition dX = Q U A, where A is the limit set, and 
/ is the domain of discontinuity for T with compact quotient B := T\Q. 
According to the conjecture of Patterson for the convex cocompact case one 
should replace by the T-submodule of distribution sections of V(o\) with 
support on the limit set A. 

The main aim of this paper is to prove the conjecture of Patterson for 
convex cocompact T C SO(l,n)o and trivial a up to two modifications which 
we will now describe. 

The first modification is that we consider hyperfunctions instead of distri- 
butions. From the technical point of view hyperfunctions are more natural and 
easier to handle. In fact, in several places we use the flabbiness of the sheaf of 
hyperfunctions. Hyperfunctions also appear in a natural way as boundary val- 
ues of eigenfunctions of Ax- In order to obtain distribution boundary values 
one would have to require growth conditions. Guided by the experience with 
cocompact groups and by the fact that the spaces of T-invariant hyperfunctions 
and distributions with support on A coincide, we believe that the cohomology 
groups are insensitive to replacing hyperfunctions by distributions (note that 
the situation is different if T has parabolic elements; see [7]). 

The second modification is in fact already necessary in the cocompact 
case to get things right at the point s = 0. Note that the principal series 
representations H a,x come as a holomorphic family parametrized by A £ C. 
In the conjecture we replace by the representation of T on the space of 
Taylor series of length k > at A of holomorphic families C 3 fi <— ► / M € 
such that supp(/ M ) C A for all jj, (this is the space C(^ i fe)C _£i '(A) below). 
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1.4. The main result. 

In the present paper we prove the conjecture of Patterson for T C S0(1, n)o 
a convex cocompact, non-cocompact and torsion-free subgroup and the trivial 
representation a of M. We restrict ourselves to this special case mainly be- 
cause of the lack of information about the Selberg zeta function in the other 
cases. 

Let us first define O^^C~' J '{A). The group G acts on the geodesic bound- 
ary dX = S 71 " 1 by means of conformal automorphisms. Let dX = Q U A be 
the decomposition of dX into the limit set A and the domain of discontinuity 

n. 

For any A G C let V\ be the representation of P on C given by man i— > 
a P-\ ._ e (/>-A) log(o) _ Let := G xp Va be the associated homogeneous 

line bundle. Note that V(-p) ^ A n-1 T£0X is the complexified bundle of 

n-l-2A 

volume forms. Moreover, V(A) = (A^T^dX) 2 (™-i) . If we choose a nowhere- 

re-l-2A 

vanishing volume form vol on dX, then vol sf™- 1 ) is a section trivializing F(A). 
Sections of V(A) can thus be viewed as functions which transform under G 
according to a conformal weight related to A. 

The union UagC VW ~^ ®X nas the structure of a holomorphic family of 
line bundles. Using the nowhere- vanishing volume form vol we define isomor- 
phisms vol n ~ 1 : V(X) = V(A + n). Hence we can identify the space of sections 
of V(A) of a given regularity with the corresponding space of sections of a fixed 
bundle, e.g., of the trivial one V{p). This allows us to speak of holomorphic 
families of sections or homomorphisms. 

By Tr x (g) : C~ w (dX, V(X)) -► C~ w {dX, V(A)), g G G, we denote the 
representation of G on the space of hyperfunction sections of V(X). As a topo- 
logical vector space C~ u '(dX, V(A)) is the space of continuous linear functionals 
on C UJ (dX, V{— A)). If g G G is fixed, then 7r x (g) depends holomorphically on 
A. 

Since A is T-invariant the space of hyper functions C~ W (A, V(A)) C 
C~ w {dX, V(A)) with support in A carries a representation of V induced by 
7r A . Let 0\C^ iAj (X) denote the space of germs at A of holomorphic families of 
sections C 3 fi i — > G C^ UJ (dX, V(/j,)) with supp(/ M ) C A. The representa- 
tion of T on that space is given by (w(g)f)n := TT^(g)f^, g G T. 

If L\ denotes the multiplication operator L\ : ^ (// — A) fc /^, k G N, 
then we have a short exact sequence 

_> O x C-"(A) 3 A C^(A) - (Aifc) C" w (A) - 

of T-modules defining C (Aifc) C7- w ( A). Note that O^C^ {K) C _a; (A, V(A)). 
Now we can formulate the main theorem of our paper. 
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Theorem 1.3. For any A G C there is k(X) G No suc/t i/iai the following 
assertions hold: 

(i) dim F*(r,0 (Aife) C^(A)) < oo /or a// k, dim iT(r, e> A C^(A)) < oo. 

(ii) X (r,O^ k) C-"(A))=0forallk. 

(iii) //* > k(X), then dim H*(T,O iw) C-» (A)) = dimtf*(I\ (A)fc) C-(A)). 

(iv) If k> k(X), then the order of the Selberg zeta function at A is given by 

(3) ord s=A Z 5 (s) = - x (r,0 A C- w (A)) 

(4) = -xi(r,o (Aifc) C7- w (A)), 

where for any T -module V with dim fP(T, V) < oo iis /irsi derived Euler 
characteristic Xi(r, V) is defined by 

n 

Xi(T,V) :=£p(-l) p dim^(r,y) . 
p=l 

It will be shown in Proposition 4.19 that one can take /c(A) := Ord^ =A ext M + 
e, where e = if A —No — p, and e = 1 otherwise, and also where ext^ is the 
extension map explained in the next subsection. In contrast to ord, Ord At=A 
denotes the (positive) order of a pole at \i = A, if there is one, and it is zero 
otherwise. 

For generic T and most A one expects fc(A) < 1. For those A one can 
replace 0^,k)C~ u '(A) by C- W (A,F(A)) and probably also by C~°°(A, V(A)), 
the space appearing in Patterson's original conjecture. 

In [38] the order of Zs(s) at s = was not given explicitly. As a corollary 
of our computations we obtain: 

Corollary 1.4. 

ord s=0 ^s(s) = dim r C~ w (A, V(0)) = dim ker(S + id) , 

where So is the normalized scattering matrix (introduced in Section 2) at zero. 

The proof of Theorem 1.3 consists of three steps. The first step which oc- 
cupies most of the paper is an explicit computation of the cohomology groups 
H*(T, (Ajfc) (7- w (A)), H*(T, OxC-^ik)). One of our main tools in these com- 
putations is the extension map which will be explained in the next subsection. 
The final results which are of interest in their own right, and much more de- 
tailed than needed for the proof of Theorem 1.3, are stated in subsections 4.3 
and 4.4. They are generalizations of our results for the two-dimensional case [6]. 
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In a second step we compare the result of this computation with the 
spectral description of the singularities of Z$ given by Patterson-Perry [38] in 
case n = 0(2), 5r < 0. Finally we employ the embedding trick in order to drop 
this assumption. The second and third steps are performed in Section 5. 

In order to compute H*(T, 0\C~ UJ (A)) we use suitable acyclic resolutions 
by T-modules formed by germs of holomorphic families of hyper functions on 
dX and fi. The proof of exactness and acyclicity of these resolutions is quite 
involved and uses some hyperfunction theory and analysis on the symmetric 
space X. It turned out that we need facts which hold true in much more 
general situations but have not been considered in the literature so far (up to 
our knowledge). This accounts for the length of subsections 4.1 and 4.2. 

1.5. The extension map. 

The present paper has a close companion [8] in which we consider the 
decomposition of the right regular representation of G on L 2 (T\G) into ir- 
reducible unitary representations in the case that T is a convex cocompact 
subgroup of a simple Lie group of real rank one. The main ingredient of both 
papers is the extension map ext^. 

Consider any T-invariant hyperfunction section / G r C~ UJ (Q,, V(A)). By 
the flabbiness of the sheaf of hyperfunction sections it can be extended across 
A; i.e., there is a hyperfunction section / G C~ UJ (dX, V(A)) which restricts 
to /. In general / is not T-invariant. Our extension map solves the problem 
of finding a T-invariant extension ext^(/). It turns out that the invariant 
extension exists and is unique for generic A. 

Now the maps ext a form in fact a meromorphic family of maps with finite- 
dimensional singularities. The highest singular part of ext at the poles of 
ext gives invariant hyperfunction (in fact distribution) sections of V(X) with 
support on A. This can be considered as a generalization of the construction of 
the Patterson-Sullivan measure which is given by the residue of ext at A = <5r- 

In Section 3 we employ a version of Green's formula in order to get 
a hold on the spaces of invariant hyperfunction sections with support on 
A. In particular, it follows that there is a discrete set of A G C, where 
/ G r C~ UJ (Q, V(X)) has to satisfy a finite number of nontrivial linear con- 
ditions in order to be invariantly extendable. To find a T-invariant extension 
of / is a cohomological problem and H 1 (T,C~ UJ (A,V(X)) essentially appears 
as its obstruction group. This is the basic observation which enables us to 
compute these cohomology groups. 

In order to provide a feeling for the extension problem let us discuss a toy 
example. We consider the extension of hyperfunctions / on R \ {0}, which 
transform as f(rx) = r x f(x) for all r G R+. Let fx be given by f\(x) := |x| A . 
For Re(A) > —1 its invariant extension as a distribution (and hence as a 
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hyperfunction) to R is just given by 

<ext A (/ A ),</>) := / 0(x)M A (ix , <f> G C C °°(R) . 

It is well-known that ext\(f\) has a meromorphic continuation to all of C with 
poles at negative integers. The residues of the continuation at these points are 
proportional to derivatives of delta distributions located at {0}. 

The construction of the meromorphic continuation of the extension map 
is closely related to the meromorphic continuation of the scattering matrix 

s x ■. r c^(n, v(\)) - r c~"(n, v(-\)) . 

If r is the trivial subgroup, then Q = dX and 5a coincides with the (normal- 
ized) Knapp-Stein intertwining operator 

J A : C~"(dX,V(X)) C^idX^Vi-X)) . 

The operators J\ form a meromorphic family and are well-studied in represen- 
tation theory. In this paper and in [8] we approach the scattering matrix S\ 
starting from J\ and using the basic identity 

Sx = res_ A J\ ext A , 

where res A denotes the restriction res A : T C- UJ (dX, V(X)) -> r C~ u >(Q,V(X)). 

In the literature the scattering matrix is usually considered as a certain 
pseudodifferential operator which can be applied to smooth, resp. distribution 
sections. A meromorphic continuation of Sx was obtained in [35] for surfaces, 
and in [40], [30] in general. In [8] we develop the theory of the scattering 
matrix and the extension map in a smooth/distribution framework for general 
rank-one spaces and arbitrary a. The main point in the present paper is the 
transition to the real analytic/hyperfunction framework. 

The main difference from most previous papers is that our primary anal- 
ysis concerns objects on the boundary dX. The spectral theory of Ay is only 
needed in some very weak form. One the other hand one can deduce the 
spectral decomposition, the meromorphic continuation of Eisenstein series and 
the properties of the resolvent kernel from the theory on the boundary. To 
illustrate this consider, e.g., the Eisenstein series. Let Px ■ C~ W (<9X, V(X)) — > 
C°°(X) denote the Poisson transform. For hefiwe define f b G r C^(0,, V(X)) 
by fb ■= E 7 er 7r A (7)((5feVol^), where S b G C^(Q, V(-p)) is the delta distribu- 
tion located at b and fi := — ^^zjy ■ Then the Eisenstein series can be written 
as 

E x (x,b) :=(P A oext A (/ 6 ))(x) . 
These applications will be contained in [8] and its continuations. 

Acknowledgement. We thank S. Patterson and P. Perry for keeping us in- 
formed about the progress of [38]. Moreover we thank A. Juhl for pointing out 
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remarks. This work was partially supported by the Sonderforschungsbereich 
288, Differential eometrie und Quantenphysik. 

2. Restriction, extension, and the scattering matrix 

2.1. Basic notions. 

The sheaf of hyperfunction sections of a real analytic vector bundle over 
a real analytic manifold is flabby. Thus the following sequence of T-modules 

-► C-"(A, V(X)) -»• c-^idx, V(A)) r ^ 2 c-^in, V(X)) 

is exact, where resn is the restriction of sections to £1. 

Let V B (X) := T\V(\)\ n . If we identify r C~ u '(0,, V(X)) C~"(B, Vb(A)), 
then resQ induces a map res A : r C , ~ a; (9X, V(A)) -► C _w (B,Fb(A)). Here 
r C _aJ (., V(A)) denotes the subspace of T-invariant sections. 

The main topic of this section is the construction of a meromorphic family 
of maps extA : C~ uj (B,Vb(X)) — > r C _w (<9X, F(A)) which are right inverse to 
resA- The poles of extA will correspond exactly to those points A € C where 
resA fails to be an isomorphism. 

The strategy of the construction of ext A is the following. We first con- 
struct ext A for Re(A) > <5r- Then we introduce the scattering matrix 5a 
: C~ U {B, V B (X)) - C-"{B, V B (-X)) by 

(5) S x := res_ A ° J\ ° ext A , 

where J A : C~ W (9X, V(A)) C' w {dX,V(-X)) is the Knapp-Stein intertwin- 
ing operator (see [26]) which we will introduce below. Assuming for a moment 
that 5r < we obtain a meromorphic continuation of S\ using results of 
Patterson [34] and [8]. Then we construct the meromorphic continuation of 
ext A by 

(6) ext A := J-x o ext_A ° S\, Re(A) < , 

where J_a (5a) is the normalized intertwining operator (scattering matrix). 

If Sr > 0, then we employ the embedding SO(l,n)o ^ SO(l,n + m)o, m 
sufficiently large, in order to reduce to the case 8r < 0. 

2.2. Holomorphic functions to topological vector spaces. 

In order to carry out the program sketched above we need to consider holo- 
morphic families of vectors in topological vector spaces. We will also consider 
holomorphic families of continuous linear maps between such vector spaces. 
Therefore we collect some preparatory material of a technical nature. All 
topological vector spaces appearing in this paper are Hausdorff and complete. 
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A holomorphic family of vectors in a locally convex topological vector 
space T denned on [/ C C is by definition a continuous function from U to T 
which is weakly holomorphic. Using Cauchy's integral formula one can show 
an equivalent characterization of holomorphic families. A map / : U — > T is 
holomorphic, if and only if for any zq G U there is a neighbourhood zq G V C U 
and a sequence {/& G J} fce N such that for z G V the sum J^k^o fk( z ~ z o) k 
converges and is equal to f(z). 

In order to speak of holomorphic families of homomorphisms from T to Q, 
where Q is another locally convex topological vector space we equip Hom(J r , Q) 
with the topology of uniform convergence on bounded sets. 

Let f :U \ {zq} — > Hom^, Q) be holomorphic and f(z) = Y^T=-N fk( z — 
zo) k for all z ^ zq close to z$. Then we say that / is meromorphic and has a 
pole of order N at zq. If k = —N, . . . , —1, are finite-dimensional, then, by 
definition, / has a finite-dimensional singularity 

Holomorphy of a map / : U — > Hom(J r , Q) can be characterized in the 
following weak form. Let Q' denote the dual space of Q with its strong topology. 
We call a subset A C T x Q' sufficiently large if for B G Hom(J r , Q) the 
condition ((f), Btp) = 0, for all (tp, <p) G A, implies i? = 0. 

Lemma 2.1. The following assertions are equivalent: 

(i) / : ?7 — > Hom(^, (?) is holomorphic. 

(ii) /:[/—> Hom(^ r , £/) is continuous, and there is a sufficiently large set 
A C J 7 x Q' such that for all (ip, (f>) £ A the function U 3 z i— > (0, f(z)ip) 
is holomorphic. 

Proof. It is obvious that (i) implies (ii). We show that (i) follows from 
(ii). It is easy to see that / is holomorphic at z if and only if 

f{z) = 2^?— dx 

for z' close to z, where the path of integration is a small circle surrounding z 
counterclockwise. Let / satisfy (ii). Then we form 

We must show that d = 0. It suffices to show that for all (ip, <fi) G A we have 
(0, d(z')ip) = 0. But this follows from (ii) and Cauchy's integral formula. □ 

Lemma 2.2. Let foil!—* Hom(J r , Q) be a sequence of holomorphic maps. 
Moreover let f : V Hom(^ r , Q) be continuous such that for a sufficiently 
large set A C T x Q' the functions (<fi, fiip) converge locally uniformly in U to 
{4>, ftp) . Then f is holomorphic, too. 
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Proof. Since the holomorphic functions {(f>, flip) converge locally uniformly 
to ((f), ftp) we conclude that the latter function is holomorphic, too. The lemma 
is now a consequence of Lemma 2.1. □ 

Lemma 2.3. Let /:{/—> Hom(JT, Q~) be continuous. Then the adjoint 
f : U Hom(5' , J 7 ') is continuous. If f is holomorphic, then so is f. 

Proof. We first show that the adjoint f':U—> Hom(Q', J 7 ') is continuous 
at zq E U. Let B C Q 1 be a bounded set. Let q be any continuous seminorm 
on J 7 '. Then we have to show that for any e > there exists 6 > such that if 
\z — Zo\ < 5, then sup^g^ q(f'(z)(p — f'(zo)<p) < e. The strong topology of J 7 ' is 
generated by the seminorms qn associated to bounded subsets D C J 7 , where 
q D (ip) := sup KeD \{iI>,k)\. We have 

saj>q D (f'(z)<i>-f'(zo)<i>) = sup \(f'(z)<j> - f'(z )<P, «)| 

0S-B <f>£B,K£D 

= sup \(4>J{z)k - f(z )K}\ 

= sup q B {f{z)K - f{zo)K) . 

Here qs is a continuous seminorm on the bidual Q" . Since the embedding 
Q <^-> Q" is continuous and / is continuous at zq we can find 5 > for any e > 
as required. 

If / is holomorphic, then holomorphy of /' follows from Lemma 2.1 when 
we take the sufficiently large set JFx Q', and use the fact that (cp, ftp) = (f'<j>, ip) 
is holomorphic for all tp G J 7 , (f) G Q' . □ 

A locally convex vector space is called Montel if its closed bounded subsets 
are compact. 

Lemma 2.4. Let F^Q^TL be locally convex topological vector spaces and 
assume that J 7 is a Montel space. If f : U — > Hom(J r , Q) and f\ : U — > 
Hom(^,H) are continuous, then the composition f± o f : V — > Hom(J 7 , 7Y) is 
continuous. If f and f\ are holomorphic, then so is the composition f± o f. 

Proof. We first prove continuity of the composition /i o / at Zq G [/. It is 
here where we need the assumption that J 7 is Montel. Let 5 C Jbea bounded 
set and s be a seminorm of TC. We have to show that for all e > there exists 
6 > such that if \z-z \ < 5, then sup^s s((f 1 of)(z)((())-(f 1 of)(zo)((f))) < e. 

Let W C U be a compact neighbourhood of zq. The map F : W x B ^ Q 
given by F(v, (p) := f(v)((p) is continuous. Since J 7 is Montel, any bounded set 
B is precompact. Thus the image B\ := F(W x i?) of the compact set W x B 
is precompact, too. In particular, B\ is bounded. Since /i(zo) is continuous 
there is a seminorm i of Q such that s(f\(zo)(ip)) < t(tp), for all tp E Q. Now 
let F C W be a neighbourhood of so small that t(f(z)(<p) — f(zo)((p)) < e/2, 
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for all z G V, for all G B, and s(/i(z)(V>) - /i(zo)(V0) < e/2, for all ^ G Bi, 
for all z G V. Then for all z G V, G B, 

s(f 1 (z)of(z)(4>)-f 1 ( Zo )of(z )(c/ ) )) < 

+*(/i(«o)[/WW-/WW]) 

< e . 

Thus we can find <5 > for any e > as required. This proves continuity of 
the composition f± o /. 

We now show that this composition is holomorphic if /, f\ are so. We 

have 

i //.Mo/W fe= i ft AW°/(») dydx . 



2m I x — z (2m) 2 II (x — z)(y — x) 

If we restrict this equation to a bounded set B C J 7 , we see as above that there 
exists some bounded set B\ C Q such that ^ x _{.^y_ x ^ (B) C B± for all y, x in 
the domain of integration. Hence we can apply Fubini's theorem to the double 
integral and obtain 

i //,W»/(^ _ i II AW°/M ixiy 



2m I x — z (2m) 2 II (x — z)(y — x) 

_ ±iMz)°m d 

2m J y — z 
= (/i <>/)(*)■ 

This shows that f± o f is holomorphic. □ 

Consider a real analytic vector bundle over a closed real analytic manifold. 
Then the spaces of real analytic, smooth, distribution, and hyperfunction sec- 
tions of the bundle equipped with their natural locally convex topologies are 
Montel spaces. 

2.3. The push-down. 

In the present subsection we define a push-down map 

7r*,_ A : C\dX, V(-X)) - C»(B, Vb(-A)), jj G {w, oo}, Re(A) > «5 r . 

The extension exi\ will then appear as its adjoint. 

Using the identification C$(B,V B (-\)) = r C^(Q, V(-X)) we want to de- 
fine 7T* _ x by 

(7) tt*,_ a (/)(6) = ^(vr- A ( 5 )/)(6), / G C«(dX, V(-X)) 

provided the sum converges. In order to prove the convergence for Re(A) > 8r 
we need the following two geometric lemmas. 

We adopt the following conventions about the notation for points of X 
and dX. A point x G dX can equivalently be denoted by a subset kM C K 
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or gP C G representing this point in dX = K/M or dX = G/P. If F C dX, 
then FM := \J kMeF kM C K. Analogously, we can denote a point b 6 X by 
a set gA" C G, where gX represents 6 in X = G/K. 

Adjoining the boundary at infinity we can consider X U dX as a compact 
manifold with boundary carrying a smooth action of G. Let r C G be a 
torsion-free convex cocompact subgroup. An equivalent characterization of 
being convex cocompact is that T acts freely and cocompactly on X U fl 

Lemma 2.5. If F c Q is compact, then (t(r n (FM)i + if) < oo. 

Proof. Note that (FM)A + K UF C A U tt is compact. Thus its intersec- 
tion with the orbit TK of the origin of X is finite. □ 

Using the Iwasawa decomposition G = KAN we write g £ G as 
g = n(g)a(g)n(g) £ KAN. The Iwasawa decomposition and, in particular, 
the maps g t— > k(<?), g i— > a(g), 5 ► n(g) are real analytic and extend to a 
complex neighbourhood of G in its complexification Gq. Let Kq, yl^ be the 
complexifications of K, A. We identify Aq with the multiplicative group C* 
such that ^4+ corresponds to [1, oo) C C*. Any g £ G has a Cartan decompo- 
sition g = ha g h' £ KA + K, where a g is uniquely determined. 

The next lemma gives some control on the complex extension of the Iwa- 
sawa decomposition. 

Lemma 2.6. Let k M £ dX. For any compact W C (dX \ k M)M 
and complex neighbourhood Sq C Kq of K there are a complex neighbourhood 
Uq C Kq of koM and constants c > 0, C < oo, such that for all g = ha g h' 
G W A + K the components K,{g~ l k) and a{g~ 1 k) extend holomorphically to 
k £ Uq, and for all k £ Uq 



Proof. The set W~ 1 koM is compact and disjoint from M. Let w £ Nk{M) 
represent the nontrivial element of the Weyl group of (g, a). Set n := 0(n), 
where 9 is the Cartan involution of G fixing K, and define N := exp(n). By 
the Bruhat decomposition G = wNP U P we have K = wk(N)M U M. Thus 
there is a compact V C N such that W^ 1 ^]^ C wn(V)M. By enlarging V 
we can assume that V is yl + -invariant, where A acts on N by (a,n) — > afia^ 1 . 
There exists a complex compact yl + -invariant neighbourhood Vq C Nq of 
V such that n(n) , a(n) , n{n) extend to Vq holomorphically. Moreover, there 
exists a complex neighbourhood Uq C Kq of k$M such that w^W^U^M C 
k(V c )M. Let k £ Uq and g = ha g h' £ WA + K. Then h~ l k = WK,(n)m for 
fi £ Vq, m £ Mq; i.e., we parametrize h~ l U c by Vq x M^;. Furthermore, 



(8) 



ca g < \a{g k)\ < Ca, 



(9) 



nig^k) £ S c . 
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a{g- l k) = aQi'^a^h^k) 

= a{a~ 1 wn{n)m) 

= a{a g n{n)) 

= a(a g nn(n)~ 1 a(n)~ 1 ) 

= a(a g na~ 1 )a(n)~ 1 a g . 
Now a g ncig l G Vq. Thus a(g~ 1 k) extends holomorphically to k G Uq. Set 

c := inf |a(n)| inf |a(n) _1 | 

C := sup \a(n)\ sup |a(n) _1 | . 

Since Vc is compact we have < c < C < oo. Then 

ca s < |a((/ _1 fc)| < Ca s . 

Now considering k we have 

= h'~ l K(a~ 1 WK(n))m 

= h'~ 1 WK(a g nn(n)~ 1 a(n)~ 1 )m 

= h'^ l wn{a g na g 1 )m . 

Since a g na~ l G Vc we see that k(<7 _1 &) extends holomorphically to k G L^. 
If we take small enough we can also satisfy h'~ 1 WK,(V(j) C Sq. Thus for a 
smaller open subset Uq C £/q we have n(g~ l k) G for all g G WA + K and 
keU C - □ 

Lemma 2.7. // Re(A) > <5r, i/ien i/ie s«m (7) converges and defines a 
holomorphic family of continuous maps 

tt*,-a : C^(dX,V(—X)) — ► C"(B, Vb(— A)), Jt G {cx),^} . 

Proof. In case Jj = oo the lemma was proved in [8]. Thus we assume 
% = uj. First we recall the definition of the topology on the spaces of real 
analytic sections of real analytic vector bundles. 

We describe C^(dX) as a direct limit of Banach spaces. Let Sq C Kq be 
a compact right M-invariant complex neighbourhood of K. Let H{Sq) denote 
the Banach space of bounded holomorphic functions on Sq equipped with the 
norm 

||/||= sup |/(*0|, f€H(S c ). 

k£S c 
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If S' c C Sq, then we have a continuous restriction TC(Sq) 7i(S' c ). Then 

C^{K) =limH(S c ) 

as a topological vector space, where the limit is taken over all compact right 
M-invariant complex neighbourhoods of K. The compact group M acts contin- 
uously on H(Sq), and this action is compatible with the restriction to smaller 
S' c . Thus the natural right action of M on C^(K) is continuous. We obtain 
C w {dX) as the closed subspace of right M-invariants in C W (K). Using the 
if -invariant volume form on dX we identify C W (<9X,U(A)) ^ C^idX). 

Next we describe the topological structure of C W (S, Vb(A)). Let {F a } be 
a finite cover of B by compact neighbourhoods which have diffeomorphic lifts 
F a C fl. Using again the if-invariant volume form we identify C u '(F a ,V(X)) 
with the direct limit of the Banach spaces 7{(Uq a ) M of M-invariant bounded 
holomorphic functions on Uq a where Uq a C Kq are M-invariant compact 
complex neighbourhoods of F a M. Any section / G C W (B, Vb(A)) defines sec- 
tions f a G C w (F a ,U(A)) by restricting the lift of /. The correspondence 
/ ^ ©o fa defines an embedding 

/ : qB,y B (A))M0q4y(A)) 

a 

as a closed subspace and, hence, a topology on C U {B, Vb(A)). 

The space C U (B, Vb(A)) depends on A. Since T acts on preserving the 
orientation the manifold B is orientable, and we can find an analytic real- 
valued nowhere-vanishing volume form vols G C W (B, Vb(— p)). Multiplcation 
by suitable complex powers of vols provides isomorphisms between any two 
bundles V B (Xi) and Vb(A 2 ). Thus we may identify ^(B, Vb(A)) with the 
fixed space C w (-B, Vb(p)) = C W (B). Using this identification we can speak of 
continuous or holomorphic families of continuous homomorphisms to or from 
C»(B,V B (\)). 

Let Q be any locally convex topological vector space, U C C be open, 
and U 3 A i— ► 4>\ G Hom(t/, C W (B, Vb{X)) be a family of continuous maps. 
This family depends continuously (holomorphically) on A, if and only if the 
composition /o^ does. 

Let F £ {-F Q } and F C be its lift. Let F C C/i C $7 be an open neigh- 
bourhood of F and TU := (dX \ U\)M. By Lemma 2.6 we can find for any 
complex M-invariant neighbourhood S'c C Kq of if a complex neighbour- 
hood Uq C Kq of FM such that a(g~ l k), n^g^k) extend to k £ Uq for all 
g G WA + K and n(g~ l k) G S'c- This allows us to define the map 

rest, c 07^(0) : H(Sc) ^ H(C/ C ) 

for all 5 G Here (vr A (< 7 )/)(A ; ) = aGr 1 *)*-'/^- 1 *)), / G H(S C ), 

k G C/ c . 
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In order to estimate the norm of res[/ c o7r A (g) we compute for / G H(Sc) 
using (8), 

||res[/ c o 7r x (g)f\\ = sup \res Uc o ir x (g)(f)(k)\ 

keU(~< 

= sup Wg^kf-Pf^g- 1 ^ 

k£U c 

< Ccfi cX - p sup |/(jfe)| 

k£S c 

= Caf eX -<>\\f\\ , 

where C is independent of k,g, f. 

By Lemma 2.5 we have (J(r \ m WA + K) < oo. Thus for almost all g £ T 
we have ||res[/ c o 7r(^)|| < Ca^ cX ~ p . Fix e > 0. Then by the definition of 5r 
the sum 

H{S C ) M sf» Yl res ^c ° nX ^)f e H(C/ C ) 

converges uniformly for Re(A) G (— oo, — <5r — e), and on the unit ball of the 
Banach space TL{Sq) m . Going over to the direct limits we conclude that 
:= J2ger res F ° ^(sO 1S a convergent sum of holomorphic maps 

{Re(A) < -S T - e} B A ^ res^ o Tr x {g) G Hon^C^X, V(A)), C W (F, V(X))) , 

which is uniformly convergent on bounded subsets of C w (dX, V(X)). Thus LT^ 
is holomorphic with respect to A by Lemma 2.2. In order to finish the proof of 
Lemma 2.7 observe that / o 7r*^ = a 11^™. □ 

For any A G C we have V(X) <g> V(-A) = A n_1 T*5X <g> C. Integra- 
tion induces a G-invariant pairing of sections of V(A) with sections of V(— A). 
Therefore we define C^idX, V(A)) := C^dX, V(-A))'. Similarly we proceed 
with Vb(A) and distribution sections of V(A) and Vb(A), respectively. 

Definition 2.8. For Re(A) > <5r and (J G {oj, oo} define the extension map 
ext A : C-i(B, V B (X)) - C"«(dX, F(A)) by ext A := < _ A . 

2.4. Elementary properties of exk\. 

Lemma 2.9. For Re(A) > 5r and j) G {cj, oo} i/ie map 

ext A G Hom(C- s ( J B,V B (A)),C , - tt (aX,y(A))) 

depends holomorphically on X. 

Proof. This follows from Lemmas 2.7 and 2.3. □ 

Lemma 2.10. Let Re(A) > <5r and G {lj,oo}. T/ien i/ie extension map 
extA /ias values in r C~^(dX, V(X)). 
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Proof. This follows immediately from the equation ir^^xi^ x (g)f) = 
7r*,_ A (/), for all / G C*(dX, V(-X)). □ 

Next we want to show that ext^ is right inverse to res^. For distributions 
this was shown in [8]. For hyperfunctions we want to argue by continuity. 
Therefore we must show that resA is continuous. This is not straightforward 
since we have defined resA as the composition of res^ and an identification, 
and there is no natural topology on C~"(£l, V(A)). 

Lemma 2.11. Let A G C. Then the restriction map 

res A : r C~"(dX, V(X)) - C~"(B, V B (X)) 

is continuous. 

Proof. We begin by giving a description of resA which is more appropri- 
ate for the present purpose. It is based on an alternative definition of the 
topological vector space C~ u '(dX, V(A)). 

Let {Fa} be a finite cover of B by compact neighbourhoods which have 
diffeomorphic lifts F a C f2. We consider the finite set L := {g £ T \ 3a, (3 such 
that gF a HFp^ 0}. Set W := clo(dX \ \J geL>a gF a ). Then any hyperfunction 
/ G C~ u '(dX, V(X)) can be represented as a sum of hyperfunctions f 9:a , fw £ 
C^idX, V(A)), g £ L, where supp(/ g , a ) C gP a , and supp(/ w ) C W. 

If / is T-invariant, then by lifting a corresponding decomposition of res\(f) 
on B to O we can choose the hyperfunctions f 9ta such that 

(10) f g ,a=nx(g)fi,a, for all a,g£L. 

This choice made, we have resx(f) = J2 a /i,a> where we view f± >a as an element 
of C-"(B, V B (X)) with support in F a . 

We now argue that resA is continuous. Consider the Frechet spaces 

g .= C- u (gF a ,V{\))(BC- u (W,V(\)) , 

geL,a 

L g : = {/ € g | f satisfies (10)} . 

Here for any closed set A C dX the space C~^(A, V(A)) is the space of hyper- 
function sections of V(A) with support on A, i.e. the topological dual of the 
space of germs at A of real analytic sections of V(— A). By the above we have 
a surjective continuous map (f> : g — > C~ UJ (dX,V(X)) with the property that 

r c-"(dx,v(x)) c<p( L g). 

There is a continuous map res : g — > C~ U} (B, Vb(A)) given by res(/) 
:= Ea /i,a- For f e L gwe have res(/)| int(Fa) = 0(/)|int(F«)- Thus r es|kcr0n^g 
= 0, and res factorizes over the Frechet space JF defined by 

.F := [0- 1 ( r C-(5X, V(A))) n L <?]/[ker(0) n L <?] . 

induces an isomorphism 4> : T — > r C~ UJ (dX,V(X)), and resA = res o cf)^ 1 . 
Hence res a is continuous. □ 
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Lemma 2.12. For Re(A) > 5r and (j G {to, 00} the map ext\ satisfies 

ies\ o ext\ = id . 

Proof. For (j = 00 this is as shown in [8, Lemma 4.5]. Since C~°°(B, Vb(A)) 
is dense in C~ U (B, Vb(A)), and reSAoextA is continuous, the assertion for (J = u; 
follows. □ 

If r C _a; (A, V(A)) = 0, then resA is injective. If in addition Re(A) > <5r, 
then we have extA o resA = id. In order to apply this observation we need 
vanishing results for r C~ w (A, V(A)). 

Lemma 2.13. // Re(A) > and Im(A) / 0, then r C^(A, V(A)) = 0. 

Remark. We will obtain much stronger vanishing results later. 

Proof. The proof is based on the fact that the Laplace-Beltrami opera- 
tor Ay is self-adjoint. If ^ (j) G r C~ u (A, V(X)), then, using the Poisson 
transform P\, we will construct a nontrivial L 2 -eigenfunction P\(f> of Ay to an 
eigenvalue \x R, obtaining a contradiction. 

We now explain the details. Let g = n(g)a(g)n(g) G KAN be the Iwasawa 
decomposition of g G G. 

Definition 2.14. The Poisson transform. 

P A : C^(3X, V(A)) C°°(X) 

is defined by 

(PA0)G?iO := / aig-'ky^mdk . 
Jk 

Here <fi G C _1 ^(9X, V(A)) is viewed as a "function" on G with values in V\ — C, 
and the integral is a formal notation meaning that the analytic functional <f> 
has to be applied to the analytic integral kernel. 

The Poisson transform Pa is continuous, G-equivariant, and 
(Ax — p 2 + \ 2 )P\4> = 0. It is injective whenever A G" — p — No- In this 
case Pa provides a topological isomorphism between C~ 0J (dX, V(X)) and 

ker ((A x - p 2 + A 2 ) : C°°(X) - C°°(X)) . 

For all these facts see [23], [18], or [45]. 

Let V C dX and U C X be such that clo(?7) n V = 0, where we take 
the closure of U in X U dX. It is not difficult to show (see Lemma 3.2, (ii) 
below) that for Re(A) > and (f> G C~ u (dX, V(A)), supp^ C V, the restriction 
of the Poisson transform P X (p to U belongs to L 2 (U). If G r C~ w (A, V(A)), 
then P\(/> is T-invariant, and therefore it descends to an eigenfunction of Ay 
in L 2 (Y). 
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Since Y is complete, the operator Ay is self-adjoint with domain 
{/ G L 2 (y)|Ay/ G L 2 (Y)}. In particular Ay cannot have nontrivial eigenvec- 
tors in L 2 (Y) to eigenvalues with nontrivial imaginary part. Since Im(A 2 ) 7^ 
we conclude that P\(ft = and hence <p = by injectivity of the Poisson 
transform. □ 

2.5 The scattering matrix. 

In this subsection we study the scattering matrix S\ defined by (5). Our 
investigation is based on a detailed knowledge on the Knapp-Stein intertwining 
operators J\ ([26], [50, Ch.lO]). In order to fix our normalization conventions 
let us first give a definition of J\ for Re(A) < 0. 

Definition 2.15. Consider / G C°°(dX, V(A)) as a right P-equivariant 
function on G with values in V x . Then J A : C°°(dX, V(A)) -► C°°(dX, F(-A)) 
is defined by the convergent integral 

(J\f)(g) ■= I f{gwn)dn . 
Jn 

For Re(A) > the operator J\ is defined by meromorphic continuation 
([26], [50, Ch.lO]). It extends continuously to C~^(dX, V(A)), ft G {00, w}, and 
this extension coincides with the adjoint of J\ acting on C^(dX,V(X)). 

To be precise at this point we must show that J\ is meromorphic in the 
sense of subsection 2.2. 

Lemma 2.16. For Re(A) < and ft G {±cj, ±00} the intertwining opera- 
tors form a holomorphic family of continuous maps J\ G Hom(C"(9X, V(A)), 
C"(9X, V(— A))). This family admits a meromorphic continuation to all of C 
having at most first order poles. 

Proof. For ft = 00 this is as shown in [8]. The case ft = — 00 follows by 
duality from Lemma 2.3. 

We now show the lemma for ft = u>. It is sufficient to prove the first asser- 
tion. The proof of the meromorphic continuation [8] (it is based on equation 
(17) ) for ft = co applies equally well to ft = to. 

Let Xi, i = 1, . . . , dim(k), be an orthonormal base of k. For any multi- 
index r = (ii, . . . , idim(k)) we se t 

dim(k) dim(k) dim(k) 

X r = I] X\\ \r\= ]T i h r\:= J] i t \ , 
1=1 1=1 1=1 

and for / G C w (K) we define the seminorm 

||/|| r = sup|/(X r fc)| . 
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For any / G C W {K) there exists an R > such that 

(11) 11/11*:= E ^rWfWr < ™ ■ 

r 

Consider the Banach space H R {K) := {/ G C W (K)\ ||/|| fl <oo}. If 0<R'<R, 
then we have an inclusion Hr(K) Hr'(K). As a topological vector space, 

C U (K) = lunH R {K) . 

Let A G C and Re(A) < 0. As usual, we identify C W (<9A, V(A)) with 
C U {K) M . We extend / G C w (i-Q to a function / A on G by 

(12) / A (fcan) = f(k)a x ^ . 
If / G C W (<9X, V(A)), then 



J x (f)(k) := / f x {kwn)dn 
,/jV 



For any Ao G with Re(Ao) < and 5 > we can find an e > such 
that for | A — Aq| < £ 



f \a{n) x °' p - a{n) x ' p \dn < 5 

JN 



holds. We then have 

\\J\ f ~ J\f\\r = sup|/ f\ (X r kwn) - f\(X r kwn)dn\ 

k&K JN 

< sup / \f(X r kwK(n))\\a(n) x °- p - a{n) x ~ p \dn 
keKJN 

< \\f\\ r f \a(n) x «~ p - a{n) x ~P\dn 

JN 

< 6\\f\\r. 

If / G 7ifi(K), then we conclude 

\\Jx f-Jxf\\R<S\\f\\ R . 

This proves continuity of the family of intertwining operators for [J = u and 
Re(A) < 0. Holomorphy now follows from [50, Lemma 10.1.3], and Lemma 2.1. 
This proves the lemma for fl = u. In the case (j = — uj we argue by duality 
using Lemma 2.3. □ 

There is a meromorphic function -P(A) such that the following functional 
equation holds: 

(13) J ^^ = 7^)- 

The function P{\) is called the Plancherel density. 
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Let 1a £ ^{dX, V(A)) be the unique if-invariant section normalized such 
that 1 A (1) = 1 (when viewed as a function on G with values in V\ = C). 

Definition 2.17. The meromorphic function c(A) is defined by 

J A l A = c(-A)l_ A . 
We define the normalized intertwining operator by 

J A := c(-A)- 1 J A . 

By (13), 

(14) ^ = c(A)c(-A) . 

This implies the following meromorphic identity: 

(15) J x o J_ A = id . 

Now we turn to the investigation of the scattering matrix S\. Unfortu- 
nately, we are not able to show directly its meromorphic continuation as an 
operator on real analytic or hyperfunction sections of V B (X). In [8] we stud- 
ied the scattering matrix S\ as an operator acting on smooth or distribution 
sections of V B (X). First we recall some of these results. 

Lemma 2.18. (i) For Re(A) > <5p the scattering matrix defined by 

S x := res_ A o J A o ext A : C~™{B, V B (X)) - C~^{B, V B (-X)) 

forms a meromorphic family of continuous maps, as does the normalized 
scattering matrix S\ := c(— A) -1 ^. 

(ii) The families S\ , S\ admit meromorphic continuations to all of C . The 
singularities of S\ and S\ in the region Re(A) < are at most finite- 
dimensional. 

(hi) The following functional equation holds: S\ o ,S_ A = id. 

(iv) The adjoint 

S' x : C°°(B, V B (X)) C°°(B, V B (-X)) 

coincides with the restriction of S\ to C°°(B,V B (X)). In particular, this 
restriction defines a meromorphic family of continuous maps 

S x : C°°(B,V B (X)) - C°°(B,Vb(-X)) • 

(v) The extension map 

ext A : C-°°{B,V B (X)) C~°°{dX,V {X)) 

has a meromorphic continuation to all ofC with at most finite- dimensional 
singularities. Moreover, ext A has values in T C~°°(dX,V(X)). 
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(vi) The equations 

ies\ o ext^ = id, 

S\ = res_ A ° J\ ° ext A , 

ext A = J-a ° ext_ A ° S\ 

extend as meromorphic identities. 

Proof. All these assertions are shown in [8]. The main point is the mero- 
morphy of the scattering matrix. The remaining assertions are easy conse- 
quences. 

A meromorphic continuation of the scattering matrix was already obtained 
in several previous papers, e.g. in [34] in case Sr < 0, and in [40], [30], [31], 
[38] in the general case. Since these papers use different conventions for conti- 
nuity we can only deduce meromorphy of matrix coefficients of the scattering 
matrix. But we can employ Cauchy's integral formula in order to obtain the 
meromorphy of the scattering matrix in the sense of the present paper. □ 

Let W C dX be a closed set with nonempty interior and consider the 
space 

(16) J~\ := {/ G C~ u (dX, V(X))\f ]w e C»(W, V(X))} . 

We equip T\ with the weakest topology such that the maps T\ 
C^(dX,V(X)) and J-\ -> C W (W,V(X)) are continuous. Using multiplication 
by suitable powers of a i^-invariant volume form we can identify J-\, A G C, 
with the fixed space T§. We employ this identification in order to speak of 
holomorphic families of vectors f\ £ J~\. 

Let F C int(VF) be closed. By resj? we denote the restriction of sections 
of V(X) to F. 

Lemma 2.19. The composition resp o J A : T\ —> C W (F, V(— A)) is well 
defined and depends meromorphically on A. 

Proof. In order to show that the composition is well defined we must show 
that if / G J^A) then J\(f) is real analytic on F. 

As a first step we reduce the proof of the lemma to the case Re(A) < 
using the translation principle. In order to write down the appropriate for- 
mulas we identify C~ U} (dX, V(X)) C~ W {K) M . The image of T x under this 
identification is independent of A and will be denoted by T . 

By [50, Thm. 10.1.5], there are nonvanishing polynomial maps b : &q — > C 
and D : a^, — ► U(g) K , such that 

(17) b(X)J x = j X -4 p oir x - 4 P(D(X)) . 
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The family of differential operators 7r x ~ 4p (D(\)) : T — > T is a holomorphic 
family of continuous maps. Thus a proof of the lemma for Re(A) < \i also 
implies a proof for Re(A) < /x + 4p. It is therefore sufficient to study J\ in its 
domain of convergence Re (A) < 0. 

We are going to decompose res^ o J A into a diagonal part j\ and an off- 
diagonal part j|. Let F\ be closed such that F C int(Fi) C F\ C int(W). 
Then there is a compact set V C N such that Fwk{N \ V)M C F 1 M. Let 
U C int(V) be such that Fwn(N\ U)M C WM, and choose a cut-off function 
X G C°°(N) such that = 1 and X|7v\y = 0. If / belongs to the dense 
subspace C w {dX) C T , then we set 

4(f)(k) := f_h(kwn)(l- X (n))dn, 
Jn 

JlUm := / f x (kwn) X (n)dn, keFM, 
Jn 

where we employ the notation fx introduced in (12). We have to show that 
these operators extend to continuous operators from T to C U {FM) M , and that 
this extension depends meromorphically on A. 
For any multi- index r, 

\\ J \{f)\\r,F = f sup 1/ fx(X r kwn)(l - x(n))dn\ 

keFM JN 

< sup \f(X r k)\ f a(n) RcX - p dn 

keWM JN 



< c 



r,W , 



where C does not depend on r and /. Recall the definition of the i?-norm (11). 
If R > is sufficiently small (depending on /), then ||«/|(/)||r,f 
< C||/||fl,w- Thus for fixed A, Re(A) < 0, there is a continuous extension 
J\ ■ T —> C W (FM) M . An argument similar to the one given in the proof of 
Lemma 2.16 shows that J\ depends holomorphically on A. 

Let ip : K/M \ M —> N be the inverse of the analytic diffeomorphism 
n i ^ wn(n)M. If / G C 1ul (dX) and x G FM, then we can write 



J \(f)( x ) = I f\(xwn)x(n)dn 

JN 

= f(xwn(n))a(n) x ~ p x(n)dn 

JN 

= [ f(xk)a(ij(k)) x+p xmk))dk . 

J K 

Let Q be defined in the same way as T (see (16)), but with W replaced by 
wk(N \ int([/)) D supp(l — x ° V 1 )- There is a continuous injection of T 
into the space of analytic functions from FM to Q given by / ^ a/, a/(x) 
:= /(x) G ^, x G FM. The multiplication by x ° V" is a continuous operator 
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m x : Q — ► C~ w (supp(x o tp)). Since the restriction of a(tp(.)) x+p to supp(xo^) 
is analytic, it defines a continuous functional on C _a; (supp(x ° V'))- Now we 
obtain a continuous extension of J| to T by 

(18) ^WW-Wl-^.mxM*))) • 

It is now easy to verify that the map 3 A i— > J| 6 Hom(jF, C W (F)) is con- 
tinuous and that for fixed / and x the function A i-> Jf(f)(x) is holomorphic. 
Thus by Lemma 2.1 the family of maps J| is holomorphic. □ 

Lemma 2.20. (i) The restrictions of S\, S\ to real analytic sections of 
V B (X) form meromorphic families of continuous maps 

S x , S x G Hom(C"(B,V B (X)),C"(B,V B (-X))) . 

(ii) T/ie singularities of S\ and S\ in the region Re(A) < are at most 
finite- dimensional. 



(iii) The adjoint 



of 



S' x : C- U {B,V B {\)) - C-"(B,V B (-\)) 



S x : C"(B, V B (X)) - C»(B, V B (-u)) 



provides a continuous extension of S\ to C U (B, V B (X)). The same holds 
true for S\ . 

(iv) The following functional equation holds on C~ U '(B,V B (—X)) : S\ o S-\ 
= id. 

Proof, (i) and (ii): Let p : 0, — > B be the projection. We choose compact 
subsets F C int(PF) C W C f2 such that = .B and define T\ as above. It 
follows from Lemma 2.18 (v), that ext A : C ,UJ (B, V B {X)) -► C^idX, V(A)) is a 
meromorphic family of continuous maps with at most finite-dimensional singu- 
larities. Since res A oext A = id we have a holomorphic family of continuous maps 
res w o ext A : C"(B, V B (X)) -► C^W, V(A)). Thus ext A : C"(B, V B (X)) -► .F A 
is meromorphic with at most finite-dimensional singularities. We can identify 
C W (S, V B (X)) with a closed subspace of C U (F, V(X)). With this identification, 
resf = res A . Assertions (i) and (ii) now follow from Lemma 2.19. 

(iii) and (iv): The adjoint S' x is the continuous extension of S\ defined 
on distribution sections. The assertions now follow from Lemma 2.18 (iv) 
and (iii). □ 

2.6. Extension of hyperfunctions and the embedding trick. 

In this subsection we want to construct the meromorphic continuation of 
the extension of hyperfunction sections of V B (X). 
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Lemma 2.21. If Sr < 0, then 

ext A G Hom(C7- w ( J B,F B (A)),C- w (aX,y(A))) 

(initially defined for Re(A) > 5r) admits a meromorphic continuation to all of 
C with at most finite- dimensional singularities. Moreover, ext\ has values in 
r C^{dX,V(X)). 

Proof. If — Re(A) > 5r, then using Lemma 2.20 (i), (iii) we define ext\ 
:= J_a o ext_A o S\. We have by Lemma 2.20 (iv): 

resA o extA = resA o J_\ o ext_A ° S\ = S-\ ° S\ = id . 

Since for < Re(A) < — 5r and Im(A) / the restriction map resA is injective 
by Lemma 2.13 and resAoextA = resA°extA = id we conclude that extA = extA- 
Thus extA is a meromorphic continuation of extA (defined on hyperfunction 
sections) . 

Note that the family extA defined on hyperfunction sections as constructed 
above is just the continuous extension of the previously obtained family extA 
defined on distribution sections. Since extA considered as an operator on distri- 
bution sections has at most finite-dimensional singularities by Lemma 2.18 (v), 
its continuous extension to hyperfunction sections has the same singularities 
which are, in particular, finite-dimensional. □ 

We now show how to drop the assumption 5r < using the embedding trick. 

Proposition 2.22. Lemma 2.21 holds true without the assumption Sr <0. 

Proof. We realize SO(l,n)o as the group of automorphisms of R™ +1 fix- 
ing the quadratic form given by diag(— 1, 1, . . . , 1). We view SO(l, n)o as the 

nx 

subgroup of SO(l, n + l)o which fixes the last element of the standard base 
of R n+2 . This embedding is compatible with the standard Cartan involution 
9 >-> t 9~ 1 - 

We set G n := SO(l,n) . Then we have a sequence G n ^ G n+l , n > 2, of 
embeddings inducing embeddings of the corresponding Iwasawa constituents 
K n ^ K n+i^ N n ^ jyn+i^ M « ^ M n+ \ and A n A n+1 . In particular we 
identify the Lie algebras a n of A n in a compatible way with R. 

The embedding G n G n+1 induces a totally geodesic embedding X n 
X n+1 and an embedding of the geodesic boundaries dX n <^-> dX n+1 . All these 
embeddings are equivariant with respect to the action of G n . 

If we view the discrete subgroup T C G n as a subgroup of G n+1 , then it is 
still convex cocompact. By Q n , f2 n+1 , A n , A™ +1 we denote the corresponding 
domains of discontinuity and limit sets. Under the embedding dX n dX n+1 
the limit set A n is identified with A™ +1 . Moreover, we have an embedding 
Q n Q n+1 inducing the embedding of compact quotients B n <^-> B n+1 . 
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The exponent of T now depends on n and is denoted by <5p. We have the 
relation 5£ +1 = 5£ - \. Thus for m sufficiently large, 5^ +m < 0. 

Let ext^ denote the extension map associated to T C G n . The aim of the 
following discussion is to show how the meromorphic continuation ext™ +1 leads 
to the continuation of ext^. 

Let P n := M n A n N n , V{\) n := G n x P n V x , and V B n(X) = T\V(X)^ nn . 

The representation V\ of P n+1 restricts to the representation K_i of P n . 

A 2 

This induces isomorphisms of bundles 

v (% + x« = ~ \) n i VWWiijn = V Bn (X - i) . 

Let 

i* : C w (B n+1 , V B n+i(X)) -+ C"(B n ,V Bn (\-±)) 

and 

j* : C w {8X n+1 , V{\) n+1 ) C w {8X r \ V(X - ^) n ) 

denote the maps given by restriction of sections. Note that j* is G n -equivariant. 
The adjoint maps define the push forward of hyperfunction sections 

U : C-"(B n ,V Bn (X)) - C-"(B n+1 ,V Bn+1 (X- i)) , 

j, : C~ u {dX n , V{X) n ) C- u (dX n+1 , V(X - . 

If G C^ UJ (B n , V B n(X)), then the push forward i*(f) has support in 

B n c £n+l g ince n+l Q = ^ 

A ~2 A ~2 

(19) supp(ext! t +l o Q((j)) c A n+1 U VL n = 8X n . 

2 

We are now going to continue ext™ using i*, ext™^l and a left inverse of j*. 

2 

As on previous occasions we trivialize V(X) n+l by powers of a K n+1 -invariant 
volume form. We thus identify C^(dX n+1 , V(X) n+1 ) with C- w (0X n+1 ) for 
all A G C. Let J7 C dX n+1 be a closed tubular neighbourhood of dX and 
fix an analytic diffeomorphism T : [—1, 1] x dX n ^ U. Then we can define 
a continuous extension t : C^idX 11 ) -► C w ({/) by (T* o t)f(r,x) := f(x), 
r € [-1, 1], x £ dX n . Let t' : C~ U (U) -► C^(dX n ) be the adjoint of i. Then 
t' o j m = id. Because of (19) we can define 

ixt^ := (t'oext"^ oi ( )(^) . 

A 2 

Then 

ext^ G Hom(C^(B n ,V B n(X)),C- UJ (dX n ,V(X) n )) 

is a meromorphic family of continuous maps with at most finite-dimensional 
singularities. 
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In order to prove that ext A provides the desired meromorphic continuation 
it remains to show that it coincides with ext A in the region Re(A) > Sfi- If 
Re(A) > #p, then Re(A) - \ > S£ +1 , and the push-down maps vr™_ A , vr™+^ +i 

are defined. It is easy to see from the definition of the push-down that 
Taking adjoints we obtain extT'+i o = o ext A . Therefore, 

2 

ext A = t' o ext"+ 1 o i, = t' o j* ° ext™ = ext™ . 

2 

It follows by meromorphy that im(ext") consists of T-invariant sections for all 
A G C. This finishes the proof of the proposition. □ 

Lemma 2.23. The identities 2.18 (vi), are valid on hyperfunction sections. 

Proof. This follows by continuity. □ 

3. Green's formula and applications 

3.1. Asymptotics of Poisson transforms. 

In this subsection we recall facts about the asymptotic behaviour of the 
Poisson transform P\<f> £ C°°(X) near dX, where <fi £ C~ U} {dX,V{X)). 

Lemma 3.1. Let dX = U U Q, where U is open and Q = dX \ U . Then 
for any closed subset F C U the composition resF o J A : C~ UJ (Q,V(X)) — > 
C U (F,V(—X)) extends to a holomorphic family of continuous maps. 

Proof. Choose a closed set W with nonempty interior such that F C 
W C U. Then C~ U} {Q,V{X)) C T\ (see (16)), and we see by Lemma 2.19 
that resi? o J\ : C^ U} (Q,V(X)) — > C U (F,V(—X)) is a meromorphic family of 
continuous maps. We argue that it is in fact holomorphic. In the course of the 
proof of Lemma 2.19 we have constructed a decomposition res^? o J\ = J} + J?, 
valid for Re (A) < 0, where J| admits a holomorphic continuation to all of C by 
(18). Since \c-^(q v(\)) = ^' we conc hide that res^o J A = J| is holomorphic. 

□ 

Lemma 3.2. (i) Let f e C°°(dX,V(X)). If Re(A) > 0, then there exists 
e > such that for a — > oo, we have 

(P x f)(ka) = a x -Pc(X)f(k) + 0{a x -p- £ ) 

uniformly in k G K. //Re(A) = 0, A / 0, then 

(Pxf)(ka) = a x -"c(X)f(k) + a- x -0Jxf(k) + 0(a^- £ ) 

uniformly in k £ K . 
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(ii) LetdX = UUQ, whereU is open and Q:=dX\U . Let f GC^(Q,F(A)). 
Then there exist smooth functions ip n on U such that 

(20) (Pxf)(ka) = a-( x+ P\j x f)(k) + ]T a~^- n ^ n {k) , k G U . 

n>l 

T/ie series converges uniformly for a>0 anc? feM in compact subsets of 
U. In particular, there exists e > such that for a — > oo, 

(P A /)(fca) = a- A ^(J A /)(A;) +0(a- A ^- £ ) 

uniformly as kM varies in compact subsets of U . 

(iii) Let U,Q be as in (ii) and f G C~ W (<9X, V(A)) siic/i i/tai rest// G 
C°°(U, V(X)). IfRe(X) > 0, £/ien i/iere exists £ > smc/i that for a — > oo, 

(P A /)(fca) = a A ""c(A)/(fc) + 0(a A ^ £ ) 

uniformly as kM varies in compact subsets of U. If Re(A) = 0, A / 0, 
£/ien t/iere exists e > stic/i i/tai /or a — ► oo, 

(P x f)(ka) = a x -?c(\)f(k) + a- A -"(J A /)(A ; ) + 0(a^- £ ) 

uniformly as kM varies in compact subsets of U. 

The asymptotic expansions can be differentiated with respect to a. 

Proof. Assertion (i) is a simple consequence of the general results con- 
cerning the asymptotics of eigenfunctions on symmetric spaces [2, Thms. 3.5 
and 3.6], combined with the limit formulas for the Poisson transform (see [45, 
Thm. 5.1.4], [49, Thm. 5.3.4]): 

hm aP- x {P x f){ka) = c(X)f(k), Re(A)>0, 

hm aP +x (P x f)(ka) = (J x f)(k), Re(A)<0. 

For / G C(dX,V(X)), supp(/) C Q, assertion (ii) follows from Theo- 
rem 4.8 in [3]. However, the proof given in that paper works as well for 
f€C-»(Q,V(\)). 

(iii) is a consequence of (i) and (ii). Indeed, let W,Wi C U be compact 
subsets such that W C int(Wi). Let x £ C%°(U) be such that x\Wi = 1- Then 
we can write / = x/ + (1 — x)fi where %/ is smooth and supp(l — x)f C 
dX \ int(Wi). We now apply (i) to xf and (ii) to (1 - x)f for kM G W. □ 

3.2. An orthogonality result. 

The following facts were shown in [8, §6]. 

Lemma 3.3. Let F C lUd be a fundamental domain for F. There exists 
a cut-off' function x G C°°(X) such that: 
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(i) There is a finite subset L C T such that supp(x) C U 9 gl 9^'i 
(") E 9 er9*X = 1; 

(iii) For any ieN and compact W C 0, sup fcgW / M) agyl+ a |V*x(& a )| < °°; 

(iv) x extends continuously to dX and the restriction \oo of this extension to 
Q is a smooth function with compact support. 

Let 4> G r C _a, (A, V(X)). Then by Lemma 3.1 the composition res_ A ° 
J x {4>) G r C7 w (0,y(-A)) = C W (B, Vb(—X)) is well-defined for all A G C. 

Proposition 3.4. // Re(A) > 0, <p G r C^(A,y(A)), and / g 
r C" w (dX, V(A)), tfien 

(res_ A o J A (»,res A (/)) = . 
Proof. At first we need the following: 
Lemma 3.5. The space 
r C^(dX, V(X)) := {/ G r C^(dX, V(X)) | /,n G C°°(tt, V(A))} 
is dense in r C~ w (dX, V(X)). 

Proof. By Proposition 2.22 the family ext^ has an at most finite-dimen- 
sional singularity at \i = X. Thus there is a finite-dimensional subspace W C 
C^iB, V B (-X)) such that (ext A )| VK x : W ± r C~ w (<9X, F(A)) is a well-defined 
continuous map, where W L := {<f> G C _a, (.B, Vb(A)) | = {0}}. Since 

C°°(B,Vb(X)) C C~^(B, Vb(X)) is dense we can choose a complement W C 
C 00 ^, Vb(A)) such that C~ U (B, V B {X)) =W ± ®W. 

Let / G T C~ UJ (dX,V(X)). Then we can write res A / = g = g L © g, 
fir 1 G W ± , g G W. Now res A (/ - ext A (5 ± )) = g. It follows that / - 
ext A (^- L ) G r C n a; (<9X,V r (A)). Let now g { G C°°(£, V B (X)) be a sequence such 
that Hindoo gi = g. Then we can decompose gi = gj~ + gi. The sections gj~ are 
smooth since gi and gi e W are so. It follows that ext A (^) G r CQ^(9X, V(A)). 
By continuity of (ext A )| W /± we have ext\(g ± ) = lim^oo ext A (^). The asser- 
tion of the lemma now follows from f = f — ext\(g ± ) + lim^oo ext A (^). 

□ 

By Lemma 3.5 it suffices to prove the proposition for / G T C^{dX, V(A)). 
By Lemma 2.13 we can assume that A G [0, oo) U iTl. 

Let A := Ax — p 2 + A 2 , P := P\ be the Poisson transform, and let \ be 
the cut-off function given by Lemma 3.3. By Br C X we denote the metric 
ball of radius R centered at the origin represented by K. 



658 



ULRICH BUNKE AND MARTIN OLBRICH 



Note that APf = and APcfi = (see subsection 2.4). Integration by 
parts gives 

(21) = (xAP<t>,Pf) L 2 {BR) -(xP^APf) L2{BR) 

= (A x Pcf>, Pf) L HB E ) - (xP<f>, APf) L 2 {BR) - ([A, x ]P<p, Pf) L 2 {BR) 

= -(v nX P& Pf)m d B R ) + (xP<P, v n pf) L 2 [dBR) 

-{[A,x]P4>,Pf)L* { B R ) , 

where n is the exterior unit normal vector field at dB B . For the following 
discussion we distinguish between the three cases: 

(i) A G (0, oo), 

(ii) Re (A) = 0, A / 0, and 

(iii) A = 0. 

We first consider the case A > 0. Lemma 3.3 (iii) implies that | [A, x]P<fiPf\ 
is integrable over all of X, and by Lemma 3.3 (ii), Lemma 3.2 (ii), (iii) and 
the T-invariance of /, cj>, 

{[A, x ]P4>,Pf)L^x) = (J2^([A,x\)P^Pf)my) = • 

7er 

Taking the limit R — ► oo in (21), and using Lemma 3.2 (i), (ii) and Lemma 3.3, 
we obtain 

= (A + p)/ Xoo(k)(Jx<l>)(k)c(X)f(k)dk 

JK 

+ (A-p)/ Xoo(k)(J x <f>)(k)c(X)f(k)dk 

JK 

= 2Xc(X) [ Xoo(k)(J\<t>)(k) f (k)dk 

JK 

= 2Ac(A)(res_ A o J A (0),res A (/)) . 

This is the assertion of the proposition for A > since c(A) / (see [19, Ch. 
IV, Thm. 6.14]). 

Now we discuss the case Re(A) = and A / 0. By Lemma 3.2 (i), the 
function P(f)(ka) now has two leading terms. Instead of taking the limit 
R — ► oo in (21) we apply lim^oo £ JJ* dR. Again we have 

lj\[A,x}P^Pf)mB R) dR = . 

The leading term a~ x ~ p J\f(k) of P(f)(ka) does not contribute to the limit 
since 

-2A lim - r R- 2X - 2 P{ X (R)JxcP,Jxf) L 2 {dx) dR = . 

r^oo r Jo y ' 
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The contribution of the term a A p c{X)f{k) leads to 

= 2Ac(A)(res_ A o J A (0), res A (/)> 

as in the case A > 0. The proposition again follows since c(A) / 0. 

Now we consider the last case A = 0. The function c(A) has a first order 
pole at A = with residue c\ / (see [19, Ch. IV, Thm. 6.14]). We redo the 
computation (21) using the Poisson transform P = P M at p in a neighbourhood 
of 0. Of course in general P M (/), P M ((/>) are not T-invariant except for p = 0. 
Nevertheless, 

^J[A, X \Pl/ R <t>,Pl/ R f)L^B R ) = 



by the theorem of Lebesgue about dominated convergence. Moreover, 



e 2 P R 



= lun((l/R + p)[ X oo(k)(J 1/R <l>)(k)c{l/R)f(k)dk 
+(1/R - p) J X o,(k)(J 1/R 4>)(k)c(l/R)f(k)dk) 

= 2 / Xoo(A;)(Jo^)(fc) Jim C ^±f{k)dk 
Jk r^oo k 

= 2ci (res o J (</>) , res (/)) . 



It follows from (21) that the latter pairing vanishes. This finishes the proof of 
the proposition in the last case A = 0. □ 

3.3. Miscellaneous results. 

The following lemma is very similar to the general result [3, Thm. 4.1]. 

Lemma 3.6. Let U C dX be a nonempty open subset. Assume that A G C 
satisfies A -p - N U -N. // <j> G C^(dX, V(X)) satisfies resu(4>) = and 
resjj o J\((f)) = 0, then = (note that (J\4>)\u is defined even if J\ has a 
pole). 

Proof. We reduce the proof to the case Re(A) > by replacing <f> by J\(4>), 
if Re(A) < 0. We can do so because A — p — No U — N and thus J\ and J_ A 
are regular and bijective (see Lemma 4.13 below). 

We now assume that Re(A) > 0. Then the Poisson transform P\ is a 
bijection between C~ w (dX, V(A)) and ker(D), where D := Ax - p 2 + A 2 . Since 
D is elliptic with real analytic coefficients P\(f> is a real analytic function. 

We argue by contradiction and assume that <j> ^ 0. Without loss of gener- 
ality we can assume that M G U. Since P\4> is real analytic and not identically 
zero the expansion (20) has nontrivial terms. Let m be the smallest integer 
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such that VVn ^ near M, where ipo := J\(f>. The contradiction will be ob- 
tained by showing that m = 0. 

With respect to the coordinates k, a the operator D has the form D = 
Do + a~ ai R(a, k), where D$ is a constant coefficient operator on A and R is a 
differential operator with coefficients which remain bounded if a — > oo (see [19, 
Ch. IV, §5, (8)]). Moreover, it is known that Dq coincides with the iV-radial 
part of D. 

We consider the iV-invariant function / G C°°{X) defined by f(na) 
:= a~( x+p+m \ Since D annihilates the asymptotic expansion (20) we have 
Df = D f = 0. On the other hand, / satisfies (A x - p 2 + (X + m) 2 )f = 0. 
Hence (A + m) 2 = A 2 . Since Re(A) > we conclude that m = 0. □ 

The following corollary is an immediate consequence of Lemma 3.6 and 
Lemma 3.1. 

Corollary 3.7. // A G" -p - N U -N, then 

res n ° A : C~ U (A, V(\)) - C"(n, V(-X)) 

is injective. 

Lemma 3.8. If X > 0, then the order of a pole q/ext M at X is at most 1. 

Proof. Let f ^ G C U (B, Vb (//)), /i G C, be a holomorphic family such that 
ext M (/ M ) has a pole of order Z > 1 at pi = X, A > 0. We assume that I > 2 and 
argue by contradiction. Let / <p G r (7~ w (<9AT, V(A)) be the leading singular 
part of ext /1 (/ M ) at /i = A. Because of res M o ext M = id we have ves\4> = and 
hence <f> G r (7~ w (A, V(A)). 

By Lemma 3.2 for any compact subset F C there exist constants 
Ci, C2, C3 such that for a » 0, A; G FM, /x near A, < p < A, 

|i^(fca)| < C ia - A ^ 

and 

(22) \(p-X) l P^xt^)(ka)\ < C 2 {(X-p) l a^ + a^) 

< C7 3 (l + log a)^a A ^ . 

In particular, P\<p G L 2 (Y). Since I > 2 we obtain by Lebesgue's theorem of 
dominated convergence 

\\P x <j>\\ 2 = {\im(p-X) l P^(f^,P x( (>) LHY) 

= J^O* ~ A )'( P M ex V(/M)' P A</>)L2(y) • 

On the other hand the estimates (22) allow partial integration, and we obtain 
for p < A, 
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(P^ext^U), P x <f>) = ^{(Ay - p 2 + A 2 )i>xt M (/,J, P X <P) 

= x^{P^tf»)i^Y-p 2 + >?)Px<t>) 
= . 

Hence ||.P\0||l 2 (y) = 0) which is a contradiction. We conclude that 1 = 1. □ 

For A G iR there is a conjugate- linear pairing V\ <8) V\ — > V_p and 
hence a natural L 2 -scalar product C°°(5, Vb(A)) C°°(S, Vb(A)) -> C. Let 
L 2 (.B, Vb(A)) be the associated Hilbert space. Using Lemma 2.18 (iv), we see 
that the adjoint with respect to this Hilbert space structure is just S-\. 

Lemma 3.9. If Re(A) = 0, then S\ is regular and unitary. 

Proof. Assume that S\ is regular. If / G C°°(B, Vb(A)), then by Lemma 2.18 

(iii), 

\\ S xf\\h(B,V B (X)) = ( S -X° S \fJ)L 2 (B,V B (X)) = ll/lli2(B,y s (A)) ■ 
Thus S\ is unitary. 

Meromorphy of S\ and ||<Sa|| = 1 for all A G ?R with S\ regular imply 
regularity of S\ for all A G zR. □ 

Lemma 3.10. // Re (A) = 0, A / 0, then 

ext x : C-"(B,V B (\)) r C~ u} (dX, V(X)) 

is regular. 

Proof. By Lemma 3.9 the scattering matrix S\ is regular for Re(A) = 0. 
Since c(— A) is regular for Re(A) = 0, A ^ 0, the same holds true for S\. Recall 
that 5a = res_A ° J\ oext^. Corollary 3.7 and the fact that the leading singular 
part of ext^ maps to hyper functions supported on the limit set A show that a 
pole of ext^ would necessarily imply a singularity of S\. Thus extA is regular 
for Re (A) = 0, A + 0. □ 

Lemma 3.11. (i) At A = the family extA has at most a first order pole. 

(ii) exto is regular if and only if So = id. 

(iii) The operator Sq — id is finite- dimensional. 

Proof. Since So is regular and unitary by Lemma 3.9 and c(A) has a first 
order pole at A = the unnormalized scattering matrix S\ has a first order 
pole at A = 0, too. The principal series representation of G on C°°(dX, V(0)) 
is irreducible ([25, Cor. 14.30] or [20, Ch.VI, Thm. 3.6]). Thus J = id by our 
choice of normalization. 



662 



ULRICH BUNKE AND MARTIN OLBRICH 



We expand J\ = (resA=oc(— A)) A -1 + and ext a = 2~3fc<o A fc ext fc + ext°, 
where J° and ext° depend holomorphically on A near A = 0. Since for k < 
we have supp(ext fc (/)) C A for all / G C~ U {B, V B (0)) Corollary 3.7 implies 
that resQ o Jg o ext fc = if and only if ext fc = 0. Since S\ has a first order pole 
it follows that ext fe = for k < — 2, hence (i). We now consider the residue of 
Sx- 

res A =oc(-A)S'o = res\ =0 S\ 

= resA=ores_A ° J\ ° ext a 

= resA=oc(— A)id + reso o jjo ext -1 . 

We conclude that resA=oc(— A)(£b — id) = reso o J °° ext -1 . The right-hand 
side of this equation is a finite-dimensional operator, and (iii) follows. Again 
by Corollary 3.7 we conclude that ext -1 = if and only if So = id. This is 
assertion (ii). □ 

Lemma 3.12. // Re(A) > 0, then the residue of ext ^ : C _a, (£, V B {n)) -> 
r C~ u (dX, V(n)) at n = A spans T C~ iAj (A, V(A)). In particular, ext M is regular 
at A if and only if r C" w (A, V(X)) = 0. 

Proof. By Lemmas 3.8, 3.11 and 3.10 for Re(/x) > the family ext M has 
poles of at most first order. Thus res /ti= Aext (U has values in r C _w (A, V(X)). 
Let d := dimim res^ = Aext^ < dim r C _a; (A, V(X)). The meromorphic identity 
res^ o ext M = id now implies that d > dimcoker resA- By Proposition 3.4 and 
Corollary 3.7, 

dim r C^(A,y(A)) > d > dim coker res A 

= dim(imres A ) ± > dim r C^(A, V(A)) . 

It follows that d = dim r C*- w (A, V(A)). This proves the lemma. □ 

Corollary 3.13. The set F := {X e C|Re(A) > 0, r C^(A, V(A)) + 0} 
is finite and contained in [0, <5r]. Moreover T is the set of singularities ofext\ 
for Re(A) > 0. If X £ J 7 then ext M has a first order pole at \x = X. 

Proof. We combine Lemmas 2.13, 3.10, and 3.12. □ 

4. Cohomology 

4.1. Hyperfunctions with parameters. 

In this subsection we compare different definitions of holomorphic families 
of hyperfunctions. Let M be a real analytic manifold and K C M be a com- 
pact subset. Then the space C'^^K) of hyperfunctions on M with support in 
K has a natural Frechet topology as the strong dual of the space of germs of 
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real analytic functions at K. Thus we can consider holomorphic families of hy- 
perfunctions in C~ U (K) as in subsection 2.2. If U C C, then by OC- w {K){U) 
we denote the space of holomorphic functions from U with values in C~ W {K). 

Hyper functions on M form a flabby sheaf Bu- Thus we can consider 
the space of hyper functions Sm(V) on an open subset V C M. This space 
does not carry a natural topology. In order to define holomorphic families 
of hyper functions on V we have to follow a different approach. Let M c be a 
complex neighbourhood of M and Om c be the sheaf of holomorphic functions 
onM c . Then by definition B M = H n M {0 Mc ), where n = dim(M) and H* M {0 Mc ) 
denotes the relative cohomology sheaf [27, p. 192ff]. It is a theorem (see [27, 
p. 219]) that {<p G B M {M) | supp(0) C K} = C^(K) in a natural manner. If 
V C M is open, then a hyperfunction / G 0m (V") can be represented (in a non- 
unique way) as a locally finite sum of hyperfunctions with compact support 
contained in V. 

Using the approach via relative cohomology we can define the sheaf of 
holomorphic families of hyperfunctions as follows. Consider the embedding 
C x M ^ C x M c . Then we define the sheaf OB M ■= W£ xM (0 C xM c ) 
on C x M. If U C C, V C M are open, then OB M {V){U) := OB M (U x 
V) is by definition the space of holomorphic families of hyperfunctions on V 
parametrized by U. 

An important consequence of this definition is the following property. Let 
T C C be Stein. Then the prescription 



defines a flabby sheaf on M (see [33, §5]). 

Let d be the Cauchy-Riemann operator acting on the first variable of 
C x M. Then we have an alternative description of the sheaf OBm as the 
solution sheaf of 8; i.e., OB M {W) = {<p e B CxM {W) \ Be/) = 0} (see [27, 
p. 308]). If K C M is compact, then we can consider 

OB M (K)(U) := {(f) e OB M (M)(U) | supp(^) C U x K} . 

Lemma 4.1. There is a natural isomorphism OC" 1 ^ '(K)(U) = OBm{K)(U). 

Proof. In this proof we employ the description of OBm as the solution 
sheaf of the partial 9-equation. First we define a map $ : OC~^ '(K)(U) — ► 
OBm(K)(U). Let {U a } be a locally finite covering of U and let {xa} be 
a subordinated partition of unity. If / € OC~ u {K)(U), then Xaf can be 
considered as an analytic functional on U x K with support in supp(xa) x K 
as follows: For <f> e (U x K) we set 




where <p x := 0(A, .) G C7 W (K). 
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Then we define <£(/) G Bq xM {U x K) to be the hyperfunction represented 
by the locally finite sum J2aXaf £ 6cxm(^ x -^0 °f hyperfunctions with 
compact support. Since the functions Xa form a partition of unity on U and 
fx depends holomorphically on A it is easy to see that dJ2 a Xaf = 0. Thus 
indeed $(/) G OB M (U x K). 

We now construct the inverse * : OB M (K)(U) -»• OC~ w (K)(U). Let 
F G OB M (K)(U). Since F G B CxM {U x M) it satisfies <9F = 0. There- 
fore the specialization F(X, .) G Bm(M) is defined for all A G f/. Clearly 
supp(F(A, .)) C -K" and we define *(/)a : = F(A, .). We must show that ^ is 
well-defined, i.e. that U 3 A i— ► G C~ w {K) is a holomorphic function. 

Prom the arguments of the proof of [22, Thm. 4.4], there exists a local 
elliptic operator J on C x M of possible infinite order acting on the second 
variable and a function u G C°°(U x M) satisfying du = such that Ju = F. 
By [22, Thm. 4.3], we can view U 3 X ^ u\ := u(X, .) as a holomorphic 
function on £7 with values in C°°(M). 

Let x £ C£°(M) be a cut-off function with x = 1 in a neighbourhood 
of X. Since J defines a continuous operator from C _w (supp(x)) into itself 
the function A i— > J(x u x) 1S holomorphic on £7 with values in C _£i, (supp(x)). 
We can write J(xu\) = F(X,.) + -Fi(A), where supp(Fi)(A) C supp(dx). 
Since K and supp(dx) are separated we have a continuous decomposition 
C~ W {K U supp(dx)) = C~ W {K) C- W (supp(dx))- Thus (/ 3 Ah F(X, .) 
is a holomorphic function with values in C~^(K). 

It remains to show that $ and $ are inverse to each other. Let F G 
OBm(K)(U). If x G C%°(U) is considered as a function on U x M, then since 
dF = the product xF £ &Cxm(U x M) is well-defined. Since supp(xi ? ) C 
supp(x) x X is compact, %.F £ C _aJ ( su PP(x) x ^Q- Using the partition of 
unity {xa} introduced above we can write F as a locally finite sum of analytic 
functionals F = J2 a XaF. 

Now we show $ o ^ = id. We have ^(F)\ = F(X, .). Then $ o ty(F) 
is given by ^2 a XaF = F. In order to prove that o <I> = id, note that 
$(/)(A, .) = J2 a XaWfx = fx- This finishes the proof of Lemma 4.1. □ 

Fix A G C. If V C M is open, then we define 

O x C-"(V) :=limOB M (V)(U) , 

where U runs over all open neighbourhoods of A. 

Lemma 4.2. The prescription M D V i-> 0\C~ u (V) defines a flabby 
sheaf 0\Bm on M . 

Proof. There exists a fundamental sequence of Stein neighbourhoods {T n } 
of A. We can write O x C- u {V) = lim OB M (V)(T n ). Recall that M D V h+ 

n 



SELBERG ZETA FUNCTION 



665 



OBM(V)(T n ) defines a flabby sheaf on M. A direct limit of flabby sheaves is 
again a flabby sheaf. □ 

For later reference we need the following result. Let 4> £ C W (M) be 
such that is a regular value. Then iV:={0 = O}cMisa real analytic 
submanifold of codimension one. Let i : N M denote the embedding. Then 
there is a natural morphism of sheaves i^OxBn — > 0\Bm- Multiplication by 
4> induces a morphism (f> : 0\Bm — ► 0\Bm- 

Lemma 4.3. The sequence of sheaves 

(23) - uO x B N - O x B M £ O x B M - 
is exact. 

Proof. Let M c be a complex neighbourhood of M such that (f> extends 
holomorphically to M c and remains to be a regular value of this extension. 
Then N c := {(f) = 0} is a complex neighbourhood of N. We use i to denote 
the embedding N c ^ M c , too. There is a natural exact sequence of sheaves 

(24) - CxMc ^ CxMc - i*0 C xA/ c - . 

Since N c C M c is a locally closed submanifold, z* = i\ is an exact functor (see 
[24, Prop. 2.5.4]). It follows that 

^Cxm(**C , CxJV c ) = **^Cx(MnAf c )( C) CxAfJ = ^CxJV^CxiVc) • 

By [27, p. 26], for p / n and g / n — 1 we have ~Hq xM (OcxM c ) = an d 
^CxAf(^CxAf c ) = 0- The long exact sequence in cohomology associated with 
(24) gives 

-> i*H^ N {0 CxNc ) -> Hc xM (0 Cx mJ ^Cxm(°CxmJ , 
or equivalently 

06 M ^ OSm . 

Taking germs at A G C we obtain (23) and conclude exactness. □ 

4.2. Acyclic resolutions. 

The main goal of this paper is to compute the cohomology groups H*(T,A4), 
where the coefficients M are certain T-modules (i.e. complex representations 
of T). A T-module M is called T-acyclic if and only if H P (T, M) = for all 
p > 1. A T- acyclic resolution of M is a complex 

C : ->■ C7° -► C7 1 ->■ C7 2 -► . . . , 

of T-modules with H°{C') = M, H P (C-) = 0, p > 1, where all (7 are T-acyclic. 
Standard homological algebra gives an isomorphism 

iT(r,M) =#*( r c-) , 
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where 

r c - . o - r C° -^ r C 1 ^ T C 2 ^ ... . 

Using suitable trivializations of the families V(X), Vb(X) we can carry over 
the results of subsection 4.1 to hyperfunction sections of V(X) and Vb(A). 

If V C dX is open, then let 0\C~ w {V) denote the space of germs at A 
of holomorphic families of hyperfunction sections of V(A) on V. By Lemma 
4.2 the prescription dX D V t— ► 0\C~^(V) defines a flabby sheaf. Let 
res n : OxC^idX) -»• A C _h, (n) be the restriction. Then we define OaC _w (A) 
:= ker res^. 

Note that A and dX are compact. By Lemma 4.1 we can identify the 
spaces 0\C~ u '(A) and 0\C~ UJ (dX) with the spaces of germs at A of holomor- 
phic functions n -► / M € C- w (A,F(/i)) and /i — > £ C^(dX,V(fi)) in the 
sense of subsection 2.2. 

Lemma 4.4. The following complex of T -modules 

(25) -► C A C- W (A) -► OxC^idX) A C _w (n) -»• 

is exact. 

Proof. The restriction res^ is surjective since the sheaf V C <9A~ 
A C~ w (V) is flabby. Exactness at the other places holds by definition. □ 

Lemma 4.5. The T-module d? A C -a, (fi) is T-acyclic. 

Proof. The group F acts freely and properly on 0. The T-module A C _W (17) 
is the space of sections of a T-equivariant flabby sheaf on Q. Thus the assertion 
can be shown by repeating the arguments of the proof of [4, Lemma 2.6]. □ 

The proof of the following proposition will occupy the remainder of this 
subsection. 

Proposition 4.6. IfX^-p- N , then OxC~ u) (dX) is F -acyclic. 

Proof. We will show that H P (F, 0\C~ UJ {dX)) = using a suitable acyclic 
resolution. If U C C is open, then define 

OC°°(X)(U) := {/ G C°°(U xX)\df = 0}, 

where d is the partial Cauchy-Riemann operator acting on the first variable. 
By [22, Thm. 4.3], OC°°(X)(U) is the space of holomorphic functions from U 
to the Frechet space C°°(X). We define 

O a C°°(X) := \unOC°°(X)(U) , 
where U runs over all neighbourhoods of A. 
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We define the operator A : OC°°(X)(U) -► OC°°(X)(U) by (Af)^ = 
(Ax — p 2 + M 2 )//i' where = /(/x, .). We use the same symbol A in order to 
denote the induced operator on 0\C°°(X). 

Note that the Poisson transform P^ comes as a holomorphic family of 
continuous maps P M : C~ u} (dX, V(jj)) -»• C ,QO (X). Viewing OxC~ u (dX) as 
a space of germs at A of holomorphic functions with values in a topological 
vector space we can define a Poisson transform P : 0\C~ bJ (dX) — ► 0\C°°(X) 
by = P„U Since (A* - p 2 + p 2 )P^ = we have A o P = 0. We need 

to know that P : A C- w (dX) ->■ {/ G O x C 00 (X) | A/ = 0} is surjective. 

PROPOSITION 4.7. Let U C C be open such that U D —p — N = 0. 
.Le£ ^ i — ^ p E U, be a continuous (smooth, holomorphic) family of eigen- 
functions, i.e., Af = 0. Then there exists a continuous (smooth, holomorphic) 
family of hyperfunctions f3f, (/3/)^ G C~ W (<9X, V(p)), such that P((3f) = f. 

Proof. In order to prove the proposition one may take any of the existing 
proofs of the pointwise surjectivity of P M ([18], [23], [46]). One has to control 
the dependence on p of the construction of the inverse map (3^. We prefer to 
follow Helgason's proof [18] because it is the most elementary one. 

Let Km denote the set of equivalence classes of irreducible representations 
of K with a nontrivial M-fixed vector. Let 8 P be the class represented by the 
representation of K on the space of homogeneous harmonic polynomials of 
degree p on R n . Then Km = {o~ P | P G No}. Hence there is a decomposition 

oo 

(26) = 

where f^ p G ker(Ax — p 2 + p 2 ) transforms according to the if -type 6 p . For 
p G" —p — No, any if -finite eigenfunction is the Poisson transform of a unique 
if -finite function on dX (see e.g. [20, Ch.III, Thm. 6.1]); i.e., 

fn, P = PfiVnv for a certain ip^ p G C°°(dX) . 

Here and in the following we identify C*(dX,V(p)) with C*(dX) using a if- 
invariant volume form. Let ij) p be the unique M-spherical function on dX of 
if -type S p , and set := P^ip p . Then we have ([18, Lemma 4.2]) 

(27) fnA ka ) = <P/iA k ) • . k £ K, a £ A . 

We want to show that for a continuous family / the series p *— ► := J2p </V>z> 
converges in the space of continuous functions from U to (dX). The 
resulting limit (3f then satisfies P(/3f) = f. In fact, we will show the stronger 
result that : {/ G C(U, C°°(X)) \ Af = 0} ->■ C(f/, C" w (aX)) is continuous. 
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Let ip £ C w (dX) and ip = J2 p fp be its decomposition with respect to 
K-types. For < R < 1 we define seminorms \ \.\\r on C^ UJ (dX) by 

oo 

II* ■■=T, R2p \\^\\h(dx ) ■ 

It is not difficult to see that the seminorms ||.||_r, < R < 1 , define the topology 
on C^idX) (compare [18, Prop. 5.2]). Thus the topology on C(U, C^(dX)) 
is given by the seminorms 

IMIw.il := sup \ \iPfi\\R , W C U compact, < R < 1 . 
new 

The function p can be represented in terms of the hypergeometric func- 
tion F = 2 Fi as follows ([18, p. 341]): 

%M = c p(v)( 1 ~ r 2 ) p -^r p F(fi + i -n + \,p + p + 1. 



2' ^ 2" r 2'r 2 -r ' 

where a G [1, oo), r = ^ and Cp(/i) := ^ p) . 

Now fix i? and a compact set C U. The following two estimates are 
crucial: 

(i) There exist constants C\ and C2 such that for all p G No and // G W 

1 < + + Jj) i-Re M < Ca(1 +p) i-Re(.) . 

In particular, 

(28) sup _L^< C (l+p) 2k 

new \CpW\ 

for some C > 0, k G N . 

(ii) Set y := -^rrx • Then there exists P 6 No such that for all p > P, /i G W, 

(29) |F( M +i,-^ + ip + p+i,y)|>i. 

Assertion (i) follows from lim^oo x a = 1 for real x and a G C (see [11, 

p. 47]). In order to verify (ii) we estimate 

1 1 1 

Qi(ji,p,y) ■= F(/j,+ + -,p + p+ -,y) -1 

1 1 In 
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for \i varying in the compact set W := (W U — W) Pi {Re(p) > 0}. By [11, 
p. 76] we have for p G W and p > sup 6l |r (Re(p) — p), 



(p,p,y)| < (| y | + l)**M+V_l|. 

r(Re(/x) + i) r(p + p-Re(p)) 



|r(M + 2)| |r(p + p-M)l p + p+\ 

If P > sup^ G ^(Re(p) — p) is large enough, then |gi(p,p, y)| < ^ for all p > P 
and jU G W. Assertion (ii) follows. 

Now we are able to estimate ||/3/||w;.R for a continuous family of eigen- 
functions /. Equations (26) and (27) imply that Q\ defined by 

Qi(/) := sup(X;^||^ p ||| 2(9x) )^ 
few P =o 

is a continuous seminorm on {/ G C(U,C°°(X)) \ Af = 0}. Using (i), (ii) and 
the fact that 5 p (Ck) = p{p + n — 2)id, where Cr is the Casimir operator of K, 
we estimate the remainder 

oo 

\\L 2 (dX) 

p=P 

(29) 00 1 1 IR 2 

< 4 J2R 2p \F(v + -, -p + -,p + p + -, ^T^ll^lliw 

(2 < 8) p 4C f(l+p)«- 

1 11 1 i?2 

•||c p (p)(l - R 2 Y'2RPF(fi + -, -p + - p + p+ - _^_)^ p ||2 2(ax) 



< C / |(l+L7^) fe ^M| 2 ^ , 



1 + i? 



1 - 
We obtain 

\\0f\\w,R<Qi(f) + C'vQ 2 (f) , 
where Q2 is the continuous seminorm on C(U,C°°(X)) given by 



Q 2 (/) := sup / \(l + C K ) k f^ka)\ 2 dk 
new \Jk 

Continuity of (3 : {/ € C(17, C°°(X)) | Af = 0} ->■ C(f/, C'^idX)) follows. 

It remains to discuss smooth and holomorphic families of eigenfunctions 
/. Let 8 be the Cauchy-Riemann operator acting on the p-variable. Let 
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/ G C°°(U, C°°(X)) be such that Af = 0. Then d k f is again a smooth family 
of eigenfunctions. Since <&^ p is holomorphic in p it follows from (27) that 

oo 

Hence J2^Lod l f., p converges for all I G No, and we conclude that 

oo oo 

(30) B k (Pf) = d k J2 <P.# = E ^V.,p = /3(5 fc /)- 

p=0 p=0 

Thus, if / is holomorphic, then so is /3f. If / is smooth, then the right-hand 
side of (30) is continuous for all k. Hence, by elliptic regularity (3f has to be 
smooth. This completes the proof of the proposition. □ 

Lemma 4.8. // A G" —p — No, then the complex 

-► O x C~ u {dX) 4 O x C°°(X) 4 O x C°°(X) -► 

is exact. 

Proof. Since is injective for /x — p — No the map P is injective. 
Exactness in the middle follows from Proposition 4.7. It remains to show that 
A is surjective. This turns out to be quite complicated. We will need the 
following technical result. 

Proposition 4.9. Let (M, g) be a connected, noncompact, real analytic, 
Riemannian manifold, and let Am be the associated Laplace operator. Let 
r G C and U C C be open. Then the operator D : C°°(U x M) — > C°°(U x M) 
given by (Df)(p,m) := ((Am + r + p?)f)(p,m) is surjective. 

Proof. We consider the adjoint operator t D:C ( : 00 (UxM)^Cc 00 (UxM). 
To prove surjectivity of D it is sufficient to show that t D is injective and has 
closed range. 

First we prove that t D is injective. Let / G C~°°(U x M) satisfy 
t Df = 0. We consider / as a hyperfunction in Buxm(U x M). The hy- 
perplanes {p = const} are non-characteristic for t D. By a theorem of Sato 
[44], f{p,m) contains the variable m G M as a real analytic parameter. Since 
supp(/) is compact we conclude by [22, Thm. 1.5], that / = 0. This shows 
injectivity of t D. 

We now prove that the range of t D is closed. Let fi G C~°°(U x M) be 
a sequence such that t Dfi =: hi converges to h G C~°°(U x M). We have to 
find / G C-°°(U x M) such that l Df = h. 

There is a compact subset T x W C U x M such that supp(ft,j) cTxlf 
for all i Let x £ C£°(U) such that X|T = 1- Since t D is of order zero with 
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respect to the first variable we have t D(xfi) = X^i = hi. Replacing /j by xfi 
and enlarging T we can and will assume that supp(/j) C T x M. We will show 
that linij^oo fi=:f exists. 

By enlarging W we can assume that any connected component of the 
closure of M \ W is noncompact. We claim that supp(/j) C T x W for all i. 
We will argue by contradiction. Let p : C x M — > M denote the projection. 
Let V be a connected component of M \ W and assume that (fi)\uxv / 0- 
Then by the unique continuation result [22, Thm. 1.5], p(supp(/j)) D V. This 
is impossible since supp(/j) and thus p(supp(/j)) are compact. It follows that 
(fi)\UxV = f° r an connected components V oi M \ W. 

Let F C M be an open subset with smooth boundary dV such that 
W C V. We choose a real nonnegative potential / F G C£°(y) with 
supp(i ? ) C V \ W. For small t we consider the Dirichlet extension Aj of the 
operator Am + i-F 1 on L 2 (V). Then A t is self-adjoint with eigenvalues < 
Ki(i) < «2(t) < «3(i) < ••• (counted with multiplicity) such that Kj(t) — ► oo 
as j — > oo for fixed i. Furthermore, the eigenvalues Kj are continuous, strictly 
increasing functions of t. We employ this fact in order to choose for any 5 £ U 
a number tg such that — (r + S 2 ) G" spec(A <4 ). 

Let H S (V) be the scale of Sobolev spaces associated with Ao- For 
s > we have H S (V) := dom(l + A ) s / 2 = dom(l + A t ) s / 2 . If s < 0, then 
H S (V) := H- S (V)'. By elliptic regularity f\ 6 R#W C C°°{V) and dually 
C-°°(V) C U se R^ s (^)- If X € C C °°(F), then multiplication by x defines a 
continuous map x '■ C°°(V) — > HseR -^(^0- 

If d spec(A <4 ), then there is a neighbourhood Zj C C \ spec(A t(5 ) of d 
such that ZjSaw (A t(S -a) _1 : n se R^ s (^) ->■ r\eR-# s (K) is a holomorphic 
family of continuous isomorphisms. We choose open neighbourhoods Us of 5 
such that — (r + /U 2 ) G for all fi E U$. 

We choose a countable set of points Sj G such that {L^-} is a locally 
finite cover of U. Let {^} be an associated smooth partition of unity. Set 
Uj := U Sj , tj := t Sj and choose x G C C °°(K> C C C °°(M) with X|vf = L Th en we 
define continuous maps 

Lj : C°°(t^ x M) — > C 00 ([/,-, f| # s (^)) C C 0O (t/ J - x V) 

seR 

by Lj(<p)(p) := (A tj . + r + / u 2 )~ 1 x0 M , where <^(m) = 0(/x,m). Note that 
G C-^iUj x V). Thus we can define the distributions /-j G C'^iUj x M) 

by := {£jhi,Lj(<l>)), for all G C 00 ^- x V); i.e., £j := 

We claim that faj = ^/j. Indeed, we have for all <p G C°°(Uj x M) 

(/m',0> = itjhuLjicf))) 

= fD&f&LM)) 
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= (fD + tjFXZjf&Ljtf)) 
= (ZjfiA)- 

Consider the set / := {j € N | Uj fl T ^ 0}. By local finiteness of the 
cover {Uj} and compactness of T the set I is finite. By the claim above we 
have fij = for j I and /• = J2jei fi,j = Ejg/%(Cj^)- Since h ^ h as 
i — ► oo and t Lj is continuous we conclude that lim^oo fi =: f exists. This 
finishes the proof of Proposition 4.9. □ 

We now finish the proof of Lemma 4.8, proving surjectivity of 

A : O x C°°(X) -► O x C°°{X) . 

Let [/] € 0\C°°(X) be represented by / G C ioc (U x X), where J7 is a precom- 
pact neighbourhood of A with smooth boundary such that U n —p — No = 
and a/ = 0. By Proposition 4.9 we can find h G C°°(l7 x X) with A/i = /. 
We have = 8 Ah = Adh. Thus 8h is a smooth family of eigenfunctions. By 
Proposition 4.7 there exists a smooth family (3{8h) £ C co (U,C~ UJ (8X)) such 
that P{f3{8h)) = dh. Here we have identified C^idX, V(n)) with C^idX), 
as usual. 

Let A [/ be the Dirichlet extension of 88* on L 2 (U). Then A^ 1 is bounded 
and can be considered MS c\ continuous operator A^} 1 : C C °°(C/) -► C°°(U). Us- 
ing that Cg([/,C- w (5X)) = Cg(?7)®C~ w (aX) we obtain an induced op- 
erator Q = ^t/ 1 ® id- Let W C W C U be & neighbourhood of A and 
choose x G C™(U) such that X|w = L We define fe : = d*Q(xP(dh)). Then 

If we define G C°°{W x X) hy <P := (/i - Pk) ]WxX , then 

00 = <9(/i - Pk)\ WxX = 8h\ WxX - {Pd)k\ WxX = 8h\ WxX - {P(38h)\ WxX = 

and A4> = A(h - Pk) [WxX = Ah {WxX = f {WxX . We let [4>] £ O x C°°(X) be 
represented by 4> and obtain A[4>] = [/]. This proves surjectivity of A and thus 
Lemma 4.8. □ 

Lemma 4.10. The T-module 0\C°°(X) is acyclic. 

Proof. If V C X is open, then let 0\C°°(y) be the space of germs at A 
of holomorphic functions with values in C°°(V). The prescription X D V i— > 
0\C°°(y) defines a soft sheaf on X. Since T acts freely and properly on X, 
and the T-module 0aC°°(X) is the space of sections of a T-equivariant soft 
sheaf on X, we can argue as in the proof of [4, Lemma 2.4]. □ 
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We return to the proof of Proposition 4.6. By Lemma 4.8 and Lemma 4.10 



is a T-acyclic resolution of O x C w (dX). 

Since O x C°°{X) = 0^ x C°°{X) we can assume that Re(A) > 0. Let 
0\C°°(Y) := r O\C°°(X) be the space of germs at A of holomorphic functions 
with values in C°°(Y) and Ay : O x C°°(Y) -► O x C°°(Y) be the operator 
induced by A. Then 



To prove Proposition 4.6 it remains to show that coker(Ay) = or, equiva- 
lently, that Ay is surjective. The proof of surjectivity of Ay is similar to the 
proof of surjectivity of A. 

Let [/] G O x C°°(Y) be represented by / G C°°(U x Y) satisfying df = 0. 
By Proposition 4.9 we can find h G C°°(U x Y) with Ayh = f. We have 
= dA Y h = A Y dh. By p* : C°°(Y) ^ C°°{X) we denote the inclusion 
given by the pull-back associated to the covering projection p : X — > Y. Then 
Ap*dh = and /3 M (p*<9/i) M G r C' u) (dX, V(fi)) is defined. The family fi ^ 
res^ o (3^(p*dh)^ G C^ U} {B, Vb{^)) is a smooth family of analytic functionals. 

Using a suitable holomorphic trivialization of the family of bundles Vb(^) 
we can identify {B ,V B {p)) with C^iB). We consider res o (5{p*dh) G 
C°°(U, C~ W (B)). As in the proof of Lemma 4.8 we solve the (9-problem Bk = 
reso /3(p*dh) for k G C°°(W, C~ W (B)), where W C W C U is a smaller 
neighbourhood of A. 

Replacing A; by k — k(X), we can and will assume that k(X) = 0. By our 
assumption Re(A) > and Corollary 3.13, ext^ is regular at /i = A or it has 
a pole of at most first order. Thus ext(fc) is smooth on W \ {A} and bounded 
on W. 

We define the family W 3 \i ^ M := - o ext„(fc M ) G C°°(T). This 
family is bounded on W, and on W \ {A} it satisfies d(f> = 0. We conclude that 
<f> is a holomorphic function from to C°°(Y). Moreover, ^4y0 = _/W x y- If 
we define [0] G O x C°°(Y) to be the element which is represented by <f>, then 
^4y[0] = [/]. This proves that Ay is surjective. The proof of Proposition 4.6 
is now complete. 

4.3. Computation of H*(F, O x C~ u '(A)). 

Let 0\C~ U {B) be the space of germs at A of holomorphic families fi ^ 
fn G C- u (B,V B (ji)) and res : r O x C~ U! (dX) -► O x C~"(B) be defined by 
res(/) M := res^. 



-► A C°°(X) 4 O a C°°(X) -► 



F u (r,0 A C^(9X)) 
H l {T,O x C^{dX)) 
H p {T,O x C-^{dX)) 



ker(^y) , 

coker(Ay) , 

0, for all p > 2 . 
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Lemma 4.11. // A -p - N , then W(r, O x C-^(A)) = for all p ^ 1 

and 

(31) H l (T, A Cr w (A)) coker(res) . 



Proof. By Lemmas 4.4, 4.5, and Proposition 4.6 the complex 

o -► o x c~ w {dx) r ^ o x c~ w (n) -► o 

is a T-acyclic resolution of OaC~ w (A). If we employ the identifications 

r c-"(n,v(p)) = c-"(B,v B ( P )) , 

then the corresponding complex of T-invariant vectors can be written in the 
form 

-► r A C^(c)X) ^ s OxC^iB) . 

We conclude that 

fl (r,O A C _w (A)) = ker(res) , 
F 1 (r,0 A C^(A)) = coker(res) , 
# p (r, OxC^iA)) = 0, for all p > 2 . 

Since res^ is injective for generic /jeCwe have ker(res) = (see [8, Prop. 6.11]). 

□ 

The aim of this subsection is to express H P (T, O x C^ ul (A)) in terms of 
spectral and topological data as eigenvalues of the Laplacian, behaviour of 
the scattering matrix and cohomology of T with values in finite-dimensional 
representations. We start with the case Re(A) > 0. 

Proposition 4.12. If Re (A) > 0, then H p {T,O x C-^(A)) = for all 
p 7^ 1 and 

J ff 1 (r,0 A C^(A)) r C- UJ (A,V(X)) 

^ f dim ker L 2 (Ay - p 2 + A 2 ) Re(A) > 0, A / 
| dim ker(So + id) A = . 

in particular, if H 1 ^, C A C _a; (A)) / 0, t/ien A G ^ C [0,<5 r ] (see CoroZ- 
/ary 3.13). 

Proof. By Lemma 4.11 it remains to compute coker(res). Since ext A has 
at most first order poles for Re(A) > and res o ext = id we can define 
ev : OxC-^(B) - r C^(A,V(X)) by ev(f) := res /1=A ext /1 (/ A ). 

We claim that the sequence 

(32) -► r A C^(dA) OxC^iB) ™ r C~ w (A, V(X)) -► 
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is exact. It is clear that res is injective. Surjectivity of ev follows from 
Lemma 3.12. Let / G kei(ev). Then ext(/) G T O x C- UJ (dX) and res o 
ext(/) = /. It remains to show ev o res = 0. Let <fi 6 r O\C~ w (dX). Then 
ext M o res^(0^) — (ftp, for all |i / A, /i close to A. It follows ev o res(0) — 
lim M ^A(/^ — A)ext M o res M ((/> /i ) = lim^A (/•* — = 0. This proves the claim. 

Exactness of (32) immediately implies coker(res) = r C~ UJ (A, V(X)). If 
Re(A) > 0, then the Poisson transform provides an isomorphism between 
r C" w (A, V(X)) and ker L2 (A Y -p 2 + X 2 ). In fact, that P x maps r C~ w (A, V(X)) 
injectively into ker L 2(Ay — p 2 + A 2 ) was already observed in the proof of 
Lemma 2.13. For the surjectivity of P\ see e.g. [8, Prop. 9.2] or [6, Lemma 
2.1]. If Re (A) = 0, A / 0, then r C^(A, V(X)) as well as ker L2 (A y - p 2 + A 2 ) 
are trivial (see Corollary 3.13 and [28], respectively). For a derivation of the 
latter fact in the framework of the present paper we refer to [8]. 

We finish the proof of the proposition by showing that r C~ w (A, V(0)) = 
im(<So — id). From the proof of Lemma 3.11 we recall the equation So — id = 
C reso o Jq o res At= oext At , where C is some nonzero constant and Jq is the 
constant term in the Laurent expansion of at 0. Now by Lemma 3.12 we 
have im(res M= oext^) = r C~ w (A, V(0)), and reso°Jo is injective by Lemma 3.7. 
Thus reso ° Jq provides the desired isomorphism. □ 

In order to compute H*(T, C A C~ aJ (A)) for Re(A) < we need detailed 
information about the singularities of the intertwining operators. 

Lemma 4.13. (i) If X £ —p — No, then the range of the Knapp-Stein in- 
tertwining operator J\ is an irreducible finite- dimensional representation 
FxofG. 

(ii) //Re(A) < and X — p — No, then J\ is an isomorphism. 
(hi) J\ has a pole if and only if X £ No, and this pole is of first order. 

(iv) We consider the following renormalized versions of the intertwining op- 
erators: I/AeN, then J x := (p - A)J M . IfRe(X) > and A N, then 
J a := J & (thus J x is regular at p = A). 

If X £ p + N , then im(J^) = ker(J_ A ) and ker(J^) = im(J_ A ). If 
Re(A) > 0, A p + No, then J x is an isomorphism. 

(v) If X £ —p — No, then the meromorphic family J x is defined by J x 
:= (p - X)- 1 ^. If Re(A) < and X -p- N , then J x := J^. 

If Re(A) < 0, then there exists a holomorphic function q x (p) which is 
defined in a small neighbourhood of X such that q x (X) / and J^oJZ^ = 
q x (p)id. 
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Proof. The lemma is a consequence of several well-known facts concerning 
intertwining operators and reducibility of principal series representations. 

By [20, Ch.VI, Thm. 3.6], the principal series representation of G on 
C- w (dX,V(\)) is (topologically) reducible if and only if A € ±(p + N ). If 
A G p + No, then by the Cartan-Helgason Theorem [19, Ch. V, Thm. 4.1], and 
Casselman's Frobenius reciprocity [49, 3.8.2], C~ u '(dX, V(A)) contains a finite- 
dimensional irreducible submodule F_ x . If Re(A) < 0, then by [49, 5.4.1.(2)], 
the range of J x is exactly the unique irreducible submodule of C~ w (dX, V(— A)) 
(the "Langlands quotient"). If A € — p — No, then this submodule is F x . This 
proves (i). If Re (A) < 0, A -p - N , then C~ u {dX, V(±A)) are irreducible 
and J\ is an isomorphism. This is assertion (ii). 

(iii) follows from [26, Thm. 3 and Prop 4.4] (see also [20, Ch. II, Thm. 
5.4]). In order to apply Prop. 4.4 (loc. cit.) we must know that -P(O) = 0. 
One can either employ irreducibility of C~ u '(dX, V(0)) and [26, 7.1], or the 
explicit formulas for c(A) (and the relation (14) P(A) _1 = c(A)c(— A) ) given in 
[19, Ch. IV, Thm. 6.14]. 

We now consider (iv) and (v). If Re(A) < 0, A —p — No, then by 
the nonvanishing result [20, Ch. II, Prop. 5.7], and the irreducibility of 
C~ UJ {dX, V(±X)) both J±* are isomorphisms for p in a neighbourhood of 
A. Thus J A o JZ X = q X (p)i& for some nowhere-vanishing local holomorphic 
function q x . 

Let now A G —p — N . By (13), 

(33) -W-i = Jr>J M = r A (/x)id, 

where 

r\p) = { ^ n "° (2) 

Since JZ X is regular and J\ is not surjective by (i), we obtain r A (A) = 0. We 
conclude that P has a pole at A if n = 0(2), and that P(\) ^ if n = 1(2). By 
[26, Sec. 12], the Plancherel density P has at most simple poles if n = 0(2), 
and it is holomorphic if n = 1(2). Thus the zero of r x at p = A is simple. Now 
(iv) follows from (33), and 



q\p):=\ 



n = 0(2) 
— n = !(2) 



is holomorphic and nonvanishing in a neighbourhood of A. This finishes the 
proof of (v). □ 

Fix A G -p - N . We define the evaluation map b : O x C~ IJJ {dX) -> F x 
by b(4>) := j\4>\. Set OlC' u {dX) := keib. Let res be the restriction of 
res : r O x C~ u '(dX) -► O x C- u {B) to r O° x C-^(dX). 
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In order to make our notation more uniform we set for Re(A) < 0, 
A0-p-N o : 

F x := 0,b:= 0,res° := res, C%C- u (dX) := O x C-"(dX) . 
Then for all A G C with Re(A) < we have an exact complex of G-modules 

(34) — ► O x C UJ (dX) — > 0\C LV (dX) —> F x — > . 

There is a well-defined map J A : O x C~ w (dX) -► 0_ x C~ w {dX) given by 
(J x f)- fi := J^ffi- It follows from Lemma 4.13 (v), that it is an isomorphism. 
Thus, by Proposition 4.6 the T-module O x C^^ \dX) is acyclic. 

Lemma 4.14. J/Re(A) < 0, then for all p > 1 

(35) H P (T, OxC^idX)) ^ H P (T, F x ) ■ 



Proof. This is an immediate consequence of (34) and the T-acyclicity of 
OlC^idX). □ 

Next we introduce a regularized scattering matrix. Let O _ x C~ UJ (B) 
:= ker(ev). Then by exactness of (32) O _ x C~ UJ (B) = im(res). We define 
S~ x : O _ x C-"(B) -► O x C~"{B) by S~ x := res o J' x o ext. Note that S~ x is 
regular at A = fi since ext M is so when restricted to ker(ev). 

Lemma 4.15. //Re(A) < 0, then coker(res°) = coker(S" A ). 

Proof. The maps 

J x : r O° x C^(dX) v O- X C~ u {dX) 

and 

res : r O_ A C^(9X) -► 0° A C ,_h '(B) 

are isomorphisms. The inverse of res o J x is given by ^xJ X ° ext (see Lemma 
4.13 (v)). Thus J- x o ext(O X C~ U (B)) = r O° x C- <JJ {dX). We conclude that 
coker(res°) = coker(5 _A ). □ 

Proposition 4.16. //Re(A) < 0, then 

(i) H°(T,O x C- u {A))=0 , 

(ii) dimil^r, O x C-"(A)) = dim coker(res) + dimfl' 1 (r,F A ) 

= dimcoker(5- A )+dimi7 1 (r,F A )-dim J H"°(r,F A ) , 

(iii) HP(T, O x C- w (K)) HP(T,F X ) for all p > 2 . 
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In particular, dim H*(T,O x C W {A)) < oo and 

X (r,0 A C- w (A)) = xF,F x ) - dimcoker(S- A ) . 

Proof. According to Lemma 4.14 the long exact cohomology sequence 
associated to (25) reads as follows: 

(36) -► r O x C- w (dX) ^ s O x C~ UJ {B) A H 1 (T,OxC~ w (A)) 

b -\ H\T,F x )^0, 

(37) -► F p (r, O x C~ u {A)) h H p (T, F x ) -► , p > 2 . 

This implies (i), (iii) and the first equation of (ii). The short exact sequence 
of complexes 

-► r O° x C' UJ (dX) O x C-"(B) -► coker(res°) -► 

| i | | 

-► r O A C^(aX) O x C~ u {B) -► coker(res) -► 

induces the exact sequence 

— > coker(i) — > coker(res°) — > coker(res) — > . 

Since O x C-' JJ {dX) is acyclic we find by (34) that coker(i) H°(T,F X ). Com- 
bining this with (36) and Lemma 4.15 we obtain the remaining assertions. 

□ 

4.4. The r -modules {x ^ k) C- u {A). 

Definition 4.17. For any fe G N we define the T-module 0( X ^C~ u) (A) as 
the quotient: 

(38) - O x C-"(A) 3 O a C- w (A) - (A)fe) C-(A) - , 

where L\ is defined by (L x f)^ := (// - A) fc / M . For any A G C define fe(A) 
:= Ord At= Aext At + e(A), where e(A) = if A — p — No, and e(A) = 1 elsewhere. 
Here Ord M= A denotes the (positive) order of a pole at jj, = A, if there is one, 
and zero otherwise. 

There are isomorphisms of T-modules (Ail) C^(A) ^ C~ 0J (A, V(X)). If 
Re(A) > 0, then by Corollary 3.13 we have k(X) < 1. 
We consider the operators 

(L k x ) p : H?(r,O x C-"(A)) -+ H*>(r,O x C-"(A)) 

induced by L x . 

Lemma 4.18. If p / 1, then (L x ) p = 0. If k > fc(A), tfiera = 0. 
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Proof. Because of the triviality of H (T,O\C W (A)) it is enough to con- 
sider the case p > 0. If A -p - N , then W(r, OxC~ w {K)) = for all p ^ 1 
by Lemma 4.11; thus (L\) p = 0. Let A G — p — No- We give F\ the structure 
of a C[L A ]-module setting L\v := 0, v G -Fa- Then 6 becomes a morphism of 
T- and C[L A ]-modules. Thus (37) is an isomorphism of C[L A ]-modules. Hence 
(L x ) p = for p > 2. 

Consider (25) as an exact sequence of T- and C[L A ]-modules. Then (31) 
and (36) become sequences of C[L A ]-modules. Let A G C with A G" — p — No- 
If [/] G fl 1 ^, C A C- W (A)) coker(res) is represented by / G O x C^(B), then 
ext o L*/ =: g G r e> A C~ w (dA:) exists for fc > Ord^ =A ext M = k{\). It follows 
that (L*)i[/] = [L\f] = [res(s)] = for k > k(X). 

Let now A G -p - N . Consider ^ G H^T, O x C~ w (A)). We employ the 
sequence (36). Since (L x )i acts trivially on H l (T,F\) we have b\ o (L x )i(4>) = 
(L x )i o 6x(0) = 0. Thus (L A )i0 = <$(/) for some / G OxC~ u {B). Suppose 
that k > Ord^ =A ext /1 + 1 = fc(A). Putting g := ext o (L^ _1 )i(/) we obtain 
(L^)i(f) = 5 o res(g) = 0. This finishes the proof of the lemma. □ 

We recall the definition of the first derived Euler characteristic Xi(F, V) 
of a T-module V: 

oo 

Xi(T,V) :=^(-lfpdim^(r,F) . 

p=0 

The following proposition contains the first three assertions of Theorem 1.3 
and the fact that equation (3) implies equation (4). 

Proposition 4.19. Let A G C. Then 

(i) dim#*(r,0 (Aifc) C^(A)) < oo. In particular, dim H*(T,C~ W (A, V{X))) 
< oo. 

(ii) x(r,o ix>k) c^(A)) = o. 

Ifk>k(X), then 

(iii) dimff*(r,0 (A)fc+1) C- w (A)) = dimF*(r,0 (Aifc) C7-(A)). 

(iv) Xl (r,o (A>fc) c- w (A)) = x (r,o A c- w (A)). 



Proof. Assertions (i) and (ii) follow from Proposition 4.16 and the long 
exact cohomology sequence associated to (38). If k > k(X), then by Lemma 
4.18 this long exact sequence splits into short exact sequences (p = 0, 1, . . .), 

o - fl*(r,o A c- w (A)) ^(r,o (A)fe) c-(A)) - ^ +1 (r,0 A C7-(A)) - . 

Now assertions (iii) and (iv) follow, too. □ 
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Proposition 4.20. If X £ C and k > k(X), then 

- dim kor L 2 (Ay — p 2 + A 2 ) Re(A) > 0, A/0 



x(r,o x c-"(A)) 
X i(r,o (A , fc) c-(A)) 



- dim ker(S + id) A = 

dim cokcr(S'" A ) + x(I\ F x ) Re(A) < . 



Proof. The proposition is an immediate consequence of Propositions 4.12, 
4.16 and 4.19. □ 

In fact, a closer study of the long exact cohomology sequence associated 
to (38) in combination with Propositions 4.12 and 4.16 gives more precise 
information about the cohomology groups H*(T, Oi x ,fc)C~ w (A)). 

Corollary 4.21. //Re(A) > 0, then set F x := 0. For all A G C and 
k £N we have the following isomorphisms and exact sequences: 

H°(T,O im C-(A)) - ker(L*)!, 
- coker(L^) 1 - H\T, {x>k) C^ '(A)) - H 2 (T, F x ) - , 
0^HP(T,F x )^HP(T,O (Xik) C^(A)) -+ (T , F x ) -+ , p>2. 

In particular, 

dim^ 1 (r,0 (Aifc) C-(A)) =dimF (r,O (Aifc) C7-(A)) + dim J ff 2 (r,F A ) 

anc? if k > k(X), then 
dimH (T,O (X:k) C~ u '(A)) = dim coker(ros) + dimif 1 (r,F A ) 

!dim r C-"(A,T/(A)) = dim ker i2 (Ay -p 2 + A 2 ) Rc(A) > 0, A/0 
dim r C""(A, V(A)) = dim ker(S'„ + id) A = 

dim coker(S- x ) +dimH 1 (r,F x ) -dhnH°(r,F x ) Rc(A) < . 



We conclude this subsection with a generalization of Lemma 3.12 to the 
case A —p — No- For k > k(X) the singular part of ext at A defines a map 
extf : O x C-»{B) -+ T 0^ k) C^{A) by 

ext<°(/) :=extoL*(/) mod L k x (O x C^(dX)) . 



Proposition 4.22. // A —p - N and k > k(X), then 



extf : O a C7-(S) r O (Aifc) C7-(A) 



zs surjective. Moreover, 

In particular, ext M is regular at X if and only if r C~ UJ (A, V(X)) = 0. 
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Proof. ext^° factorizes over coker(res), and since resoext = id, this factor- 
ization is injective. By Corollary 4.21, dim coker(res) = dim r 0^x,k)C~ U} (A). 
This implies surjectivity of ext^°. Again by Corollary 4.21 it follows that 

dim r (7~ w (A, V(\)) = dim ker(L\ : coker(res) — > coker(res)) . 

The remaining assertions of the proposition are now obvious. □ 

Remark. Since in general for A £ —p — No we have dim H 1 (T , F\) ^ 0, 
the map ext^° is not surjective in view of the formula 

dim r O (Aife) C^(A) = dim coker(res) + dimi7 1 (r, F\) . 



5. The singularities of the Selberg zeta function 

5.1. The embedding trick. 

Let Zs(X) denote the Selberg zeta function associated to T introduced in 
subsection 1.1. For dimpf) even the spectral description of its singularities was 
worked out in Patterson- Perry [38]. This description simplifies considerably if 
we assume that 5r < 0. 

We are going to prove the remaining assertion (iv), equation (3), of The- 
orem 1.3 in two steps: 

(i) First we employ the embedding trick (which was already used in the proof 
of Proposition 2.22) in order to show in Corollary 5.5 that the equality 
(3) under the additional assumptions 5r < and dim(X) = 0(2) implies 
(3) in general. 

(ii) Then we prove (3) under the additional assumptions 5r < and dim(X) = 
0(2). 

In the present subsection we are concerned with step (i). 

We adopt the notation G n , dX n , etc. as introduced in the proof of Propo- 
sition 2.22. 

Proposition 5.1. 

(39) X (r, OaC- w (a»)) = X (r, o x _ L c^(A n+1 )) - x (r, o A+i c-(A" +1 )) . 



Proof. We will construct an exact sequence of T-modules 
(40) A C^(A n ) h O x _iC-^(A n+1 ) ^ O x+ iC-"(A n+1 ) . 
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Here is the push forward associated to the embedding dX n <^-> X n+1 (see 
the proof of 2.22). The map $ is multiplication by a ^"-invariant real analytic 
section <j) of V(p n+1 + l) ra+1 — > (9X ra+1 which vanishes on <9X ra of first order. 
We will construct in Lemma 5.2 and show exactness of (40) in Lemma 5.3 
below. Then Proposition 5.1 will follow in view of the long exact sequence in 
cohomology associated to (40) and the fact that the cohomology groups are 
finite-dimensional. 

Lemma 5.2. There exists a real analytic function (f> G C u) (G n+1 ) which 
satisfies: 

(i) {0 = 0} = G n M n+1 A n+1 N n+1 , 

(ii) <P(hg) = 4(g), for all heG n ,g£ G n+1 , 

(iii) 4>(gman) = a(f)(g), for all man G M n+1 A n+1 N n+1 . 

(iv) If (j) is considered as a section of V(p n+1 + l) n+1 , then it vanishes on 
dX n of first order. 

Proof. We consider the standard representation ir of G n+1 = 
SO(l,n + l)o on R n+2 . Let {ei\i = 0, . . . ,n + l} be the standard base and (., .) 
be the G n+1 -invariant bilinear form diag(— 1, 1, . . . , 1). We specify the Iwasawa 
decomposition by K n+1 := {g G G n+1 \ ge = e } = SO(n + 1) and 

( / cosh(t) sinh(t) \ 
A + := \ sinh(t) cosh(t) | * > 

1 V id Rn J 

Then we define <p(g) := (e n +i,ir(g)(eo + ei)). Since e„+i is ^"-invariant, 
eo + ei is M n+1 -invariant and the highest weight vector with respect to A + , the 
assertions (ii) and (iii) follow. Thus we can consider as a G n -invariant section 
of V(p n+1 + l) n+1 . In the usual trivialization by the K n+1 -invariant section 
it is simply given by the height function cp(x) = x n +±, x = (x\, . . . , x n+ \) 
G S n = dX n+1 . This is easily seen using the K n+1 -equivariant embedding 
S n 3 x h+ (l,x) G R"+ 2 . We conclude that {x G dX n+1 \ <p(x) = 0} = 
{x G dX n+1 1 x n+ \ = 0} = dX n , and that <fi vanishes on dX n of first order. □ 

We now define $ M : C w (dX n+1 , V(p) n+1 ) -► C w (dX n+1 , F(u + l) n+1 ) by 
: = <f>{g)f(g)- Indeed by Lemma 5.2 (iii), 

$ M (/)( 5 mara) = <f>(gman) f (gman) = a4>(g)a»- pn+1 f(g) = a^ +1 ^" +I $^(/)(a) , 

for all 5 G G n+1 , man G M n+1 A n+1 iV n+1 , and thus $(/) is a section of 
+ l) n+1 . By Lemma 5.2 (ii), the map <J>^ is G n -equivariant. ex- 
tends to hyperfunction sections and satisfies supp(<I> M /) C supp(/), for all 
/ G C- u (dX n+1 ,V(ji) n+1 ). 
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We define $ : (9 A _i C~ w (A n+1 ) - C A+ i C^(A"+ 1 ) by (<t>j% := 
Lemma 5.3. T/ie sequence (40) is exact. 

Proof. Using Lemmas 4.2 and 4.3 we obtain the following diagram: 


-> A C*-"(aX n ) ^> C A _iC*-"(9X n+1 ) % O x+ iC-"(dX n+1 ) -> 

J- J- X ) 

o e> A c-*'(cr) ^ o x _iC-"(n n+1 ) o x+ iC-^(n n+1 ) -> o 



where the rows are exact, and the columns are surjective. The sequence (40) 
is just the complex of kernels of the columns of the diagram above, and this 
complex is exact by the Snake lemma. This finishes the proof of both the 
lemma and Proposition 5.1. □ 

The significance of Proposition 5.1 is that the numbers %(r, 0\C~ UJ (A n )) 
behave in the same way with respect to embeddings G n <^-> G n+1 as the orders 
of the singularities of the Selberg zeta function Z$- Let V C G n be convex 
cocompact. If we consider T as a convex cocompact subgroup of G n+l , then 
Zs : n+i(s) denotes the corresponding Selberg zeta function. The following ele- 
mentary fact was noted in [34]. 

Lemma 5.4. 

oo 

Zs,n+l(s) = J] Z S,n( s + 3 + 7Z) 
3=0 1 

and, consequently, 

oo 

(41) ord s=x Z s ,n+i(s) = J2 ord s=\+j+± Z s,n( s ) , 

ord s=A ^s,n(s) = ord s=A _iZ 5jn+ i(s) - ord s=A+ iZ 5jn+1 (s) . 
Combining equation (41) with Proposition 5.1 we obtain: 

Corollary 5.5. Let T C G n be convex cocompact. If equation (3) 
holds true for T viewed as a subgroup of G n+1 , then so does it for T viewed 
as a subgroup of G n . In particular, it is sufficient to prove (3) under the 
assumptions 5r < and dim(X) = 0(2). 

5.2. Singularities and cohomology. 

In this subsection we prove Theorem 1.3 (iv), (3), under the assumptions 
that n = 0(2) and 5r < 0. In [38] the order of the singularities of the Selberg 
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zeta function was expressed in terms of traces of residues of certain meromor- 
phic families of operators. Our main task is to link this description with our 
computation of x(I\ e> A C^(A)). 

First observe that (3) holds true for Re(A) > 0. Indeed because of our 
assumption Sr < on the one hand the infinite product (2) defining the Selberg 
zeta function Zs{\) converges and thus ord M=A Zs(/x) = 0. On the other hand 
X(I\ O a C- w (A)) = by Proposition 4.12. 

It remains to prove (3) for Re(A) < 0. First we give an expression for 
dim coker(S'- A : O a _ x C- w {B) -► O x C^(B)) in terms of the trace of the 
residue of the logarithmic derivative of S~ x . Because of the assumption Sr < 
we have O _ X C~"(B) = 0^ x C~^(B). 

In the following we review results of Patterson-Perry [38]. We fix an 
analytic Riemannian metric on B in the canonical conformal class. This metric 
induces a volume form which we employ in order to identify all bundles Vb(A) 
with B x C = Vb(p). Then the scattering matrix becomes a germ at A of a 
meromorphic family of operators S~ x on C*(B), * = ±cj, ±oo. 

Let As be the Laplace operator on B associated to the Riemannian metric. 
Viewed as an unbounded symmetric operator on the Hilbert space TL := L 2 (B) 
the sum A# + 1 is positive. Let P := y / A~b~+T be the positive square root de- 
fined by spectral theory. Then P := VA_b + 1 is a pseudodifferential operator 
of order 1. In particular, P and its complex powers P^ act on C ±00 (B). For 
fi close to A the scattering matrix can be factorized as 

(42) SI^P-^id + Ki-^p-* , 

where K(—(i) is a holomorphic family of pseudodifferential operators belonging 
to the (n + l) th Schatten class (i.e. K{— /i) n+1 is of trace class). The inverse 
(1 + K(— /x)) _1 is a meromorphic family of operators with finite-dimensional 
residues. 

Proposition 5.6. 

dimcoker (V A : 0_ A C^(S) -► O a C _w (B)) = Tr res M= _ A (l+^(/i))- 1 i^'(^) , 
where K'(fi) denotes the derivative of K((i) with respect to fi. 

Proof. Let 0\H denote the space of germs at A of holomorphic families 
of vectors in H and let (1 + K) : CL A W -► 0\H be given by ((1 + K)f)^ 
:= (1 + / € 0_ A W. 

Lemma 5.7. 



dim coker(S' A ) = dim coker(l + K). 
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Proof. For k £ N we define 

(Aifc) C- w (B) := coker(L* : O x C^(B) - A C—(£)) . 

Let 5(" A ' fc ) : ( _ A)fc) C- w (S) -► 0^ k) C~ u {B) be the operator induced by 
S- A . Fix jfe > fc(A). If / € im(L k x ), then g := jxS x (f) G 0_aC- w (B) is 
defined and / = S~ x (g) (see Lemma 4.13 (v)). It follows that L\ acts trivially 
on coker(S ,_A ). Hence we can identify coker(»S _A ) with coker(S'^ A ' fe )) in the 
natural way. As spaces of finite Taylor series with values in a Frechet space, the 
spaces 0( ±Aifc )C _w (-B) are Frechet spaces, too. By Lemma 2.20 the map S^~ X ' k ^ 
is continuous with finite-dimensional cokernel. Hence it has closed range by 
the open mapping theorem. Thus the induced topology on coker(S ,_A ) = 
coker(S(" A,A: )) is Hausdorff. 

Let 0\C~°°(B) denote the space of germs at A of holomorphic families of 
distributions on B, and let 0(x,k)C~°°(B) and O^x^TL be the corresponding 
quotient spaces of finite Taylor series. We define 

P : A C™(£) -► O x C~°°(B) 

by (Pf)fi = P~^f^. We denote the induced operator on Ot Xik \C~°°(B) by the 
same symbol. The composition 

P : O iKk) H ^ O iXtk) C-°°(B) 4 0(x,k)C~°°{B) ^ {Xtk) C~ u {B) 
has dense range. 

Now let h £ OxU. We claim that h £ im(l + K) if and only if Ph 
considered as a hyperfunction is in the image of S~ x . Indeed, let Ph = S~ x j 
for some j G C A C^(S). Then -p ^ := (1 + ^(/x))- 1 ^) defines a 
meromorphic family of vectors in 7i. It is regular at \i = A since P^(g^^) = 
by (42) and P M is injective. Vice versa, let h = (1 + K)g for some g £ 0- X TL. 
The family := P M (<7_^), \i near A, defines an element j £ 0- X C~°°(B) 
satisfying Ph = S~ x j. 

The above claim implies that p induces a map : coker(l + K) — > 
coker(S ,_A ) which is injective. Moreover, it has dense range. Since coker(»S _A ) 
is finite-dimensional and Hausdorff p* must be surjective. The lemma follows. 

□ 

The following lemma is known in one form or another (compare [15]). For 
completeness we include a proof here. 

Lemma 5.8. 



Tr res M= _ A (l + K {/J,))' 1 K' (/x) = dim coker(l + K) 
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Proof. Set s := —A. Let P(fJ>) be the holomorphic family of finite-dimen- 
sional projections given by 



where the path of integration is a small circle enclosing G C counterclockwise 
and \x is close to s. There is a holomorphic family of invertible operators 



U(n) ([43, Thm. XII.12]) such that U^)" 1 P((j.)U(fi) = P(s). We define 
T(ji) := Uiny^l + KinfiUin). Then 

T(p)P(s) = U(jj,)- 1 (1 + K(ij,))P(jj)U(jj,) 



= U^)- 1 P{^){1 + K(ti))U(») = P(s)T^) . 



Let V := P(s)H and W := (1 - P(s))W. Then 



Trres^T^)- 1 ^^) = Tr res^ =s (l + K fa))- 1 K' (») . 



-U(fi)- 1 U'( t i)U( f i)- 1 (l + Kfa))Ufa) . 



Using the facts that all singular terms of (1 + K(fi)) 1 are finite-dimensional 
and that the trace is cyclic, we compute 



Trres^T^T'Gu) 
= Trres^ =s ^)- 1 (l+^))- 1 f/( /[ /) 

-u^r'u'^u^r 1 ^ + K(fi))u (fi)] 

= Tr res^ =s 

p^u'in) + u^r\\ + KMr^K'Muip) 

= Tvves^ =s (l + K(fi)y 1 K'( f i) . 



Let T:O s H^ O s H be given by (T/)(/x) = T(/i)/(/i), / G O s H. Then 





where B{(i) is invertible for /i — s small. 
We claim that 



In fact 



This proves the claim. 



we have 



dim coker(T) = dim coker(l + K ) . 
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Now 

lVres^ =i T(/i)- 1 r'(/i) = Tr ies IJi=s A(p 1 )~ 1 A' (p) 

for the holomorphic family of operators A(p) on the finite-dimensional space 
V. Moreover dim coker(T) = dim coker(^4), where A : O s V — > O s V is given by 
(Af)(p) := A(p)f(p), f e O s V . In order to finish the proof of the proposition 
we must show that 

Tr ies lx=s A{p)~ 1 A' {p) = dim coker(^) . 



Now 



Trres^A^)- 1 ^'^) = res M=s Tr A(p)- l A'(p) 

det(A(/i))' 



= res. 



M_s det(A(/i)) 
= ord M=s det(A(/i)) . 

By Gauss's algorithm ^4(/u) can be transformed to a holomorphic family of di- 
agonal matrices A(p) through multiplication from the left and right with holo- 
morphic matrix functions with invertible determinants. We have dim coker(A) 
= dim coker(A), where A : O s V -»■ O s V is given by (Af)(p) := A(fj,) f '(//) , 
/ G O s V, and ord M=s det(^4(/x)) = ord /J=s det(A(/x)). But for holomorphic 
families of diagonal matrices the equation dim coker^L = ord M=s det(A(fi)) is 
obvious. This finishes the proof of the lemma. □ 

Proposition 5.6 follows from Lemmas 5.7 and 5.8. □ 

We now recall the description of the singularities of the Selberg zeta 
function Zs(X) for Re(A) < given in [38]. Note that our standing hypothesis 
is n = 0(2) and <5r < 0. This simplifies things considerably because the 
point spectrum of Ay is absent. Let Re(A) < and set n\ 
:=Trres M= _ A (l+^(/i))- 1 K / (/i). Then 



oy& s= \Z s {s) 



n\ A -p - N 

n x - X {Y) dim F x A G -p - N . 



Since Y has the homotopy type of a finite CW-complex we have %(y, -P\) = 
X(Y) dim F\. Equation (3) now follows from Propositions 5.6 and 4.20. 

By Corollary 5.5 a proof of (3) under the assumptions n = 0(2) and 5r < 
implies (3) without these assumptions. This finishes the proof of Theorem 1.3. 
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